Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
bundle:
name: bind-undeployed-dependency

resources:
jobs:
deployed:
name: deployed upstream
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
bundle:
name: bind-undeployed-dependency

resources:
jobs:
deployed:
name: deployed upstream

new_upstream:
name: new upstream

job_1:
name: bound job
description: "runs after ${resources.jobs.deployed.id} and ${resources.jobs.new_upstream.id}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Cloud = false
EnvMatrix.DMS = ["", "true"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@

=== Deploy the bundle with only the upstream job

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/bind-undeployed-dependency/default/files...
Created jobs.deployed
Files: 2 uploaded, 0 deleted
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged

=== Add a job that references both the deployed job and a job that is not deployed yet

=== Bind without auto-approve shows the changes and does not touch the state

>>> errcode [CLI] bundle deployment bind job_1 [EXTERNAL_JOB_ID]
Plan: update resources.jobs.job_1

Changes detected:
~ deployment: null -> {"kind":"BUNDLE","metadata_file_path":"/Workspace/Users/[USERNAME]/.bundle/bind-undeployed-dependency/default/state/metadata.json"}
~ description: null -> "runs after [DEPLOYED_ID] and ${resources.jobs.new_upstream.id}"
~ edit_mode: null -> "UI_LOCKED"
~ format: null -> "MULTI_TASK"
~ max_concurrent_runs: null -> 1
~ name: "external job" -> "bound job"
~ queue: null -> {"enabled":true}

Error: this bind operation requires user confirmation, but the current console does not support prompting.
To proceed, use --auto-approve after reviewing the plan above.


Exit code: 1

>>> jq .state | keys .databricks/bundle/default/resources.json
[
"resources.jobs.deployed"
]

=== Bind with auto-approve records the job and its dependencies

>>> [CLI] bundle deployment bind job_1 [EXTERNAL_JOB_ID] --auto-approve
Successfully bound job with an id '[EXTERNAL_JOB_ID]'
Run 'bundle deploy' to deploy changes to your workspace

>>> jq {serial, job_1: .state["resources.jobs.job_1"] | {__id__, depends_on, description: .state.description}} .databricks/bundle/default/resources.json
{
"serial": 3,
"job_1": {
"__id__": "[EXTERNAL_JOB_ID]",
"depends_on": [
{
"node": "resources.jobs.deployed",
"label": "${resources.jobs.deployed.id}"
},
{
"node": "resources.jobs.new_upstream",
"label": "${resources.jobs.new_upstream.id}"
}
],
"description": "runs after [DEPLOYED_ID] and ${resources.jobs.new_upstream.id}"
}
}

>>> ls .databricks/bundle/default
deployment.json
resources.json
sync-snapshots

=== Deploy creates the new upstream and updates the bound job

>>> [CLI] bundle plan
update jobs.job_1
create jobs.new_upstream

Plan: 1 to add, 1 to change, 0 to delete, 1 unchanged

>>> [CLI] bundle deploy
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/bind-undeployed-dependency/default/files...
Created jobs.new_upstream
Updated jobs.job_1
Files: 1 uploaded, 0 deleted
Resources: 1 created, 1 changed, 0 deleted, 1 unchanged

>>> [CLI] jobs get [EXTERNAL_JOB_ID]
runs after [DEPLOYED_ID] and [NEW_UPSTREAM_ID]

=== Nothing is left to deploy

>>> [CLI] bundle plan
Plan: 0 to add, 0 to change, 0 to delete, 3 unchanged
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
title "Deploy the bundle with only the upstream job"
trace $CLI bundle deploy

title "Add a job that references both the deployed job and a job that is not deployed yet"
cp databricks_v2.yml databricks.yml
job_id=$($CLI jobs create --json '{"name": "external job"}' | jq -r '.job_id')
add_repl "$job_id" EXTERNAL_JOB_ID

title "Bind without auto-approve shows the changes and does not touch the state"
trace errcode $CLI bundle deployment bind job_1 $job_id
trace jq '.state | keys' .databricks/bundle/default/resources.json

title "Bind with auto-approve records the job and its dependencies"
trace $CLI bundle deployment bind job_1 $job_id --auto-approve
trace jq '{serial, job_1: .state["resources.jobs.job_1"] | {__id__, depends_on, description: .state.description}}' .databricks/bundle/default/resources.json
trace ls .databricks/bundle/default

title "Deploy creates the new upstream and updates the bound job"
trace $CLI bundle plan
trace $CLI bundle deploy
trace $CLI jobs get $job_id | jq -r .settings.description | sed "s/$(read_id.py deployed)/[DEPLOYED_ID]/; s/$(read_id.py new_upstream)/[NEW_UPSTREAM_ID]/"

title "Nothing is left to deploy"
trace $CLI bundle plan
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Cloud = false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] The script creates .databricks/bundle/default/ via bundle deploy and bundle deployment bind, but no ancestor test.toml ignores .databricks (the parent chain only has Ignore = ["databricks.yml"] and the framework auto-adds .cache). After the script finishes the harness lists the temp dir (acceptance_test.go:1157) and flags every file not in inputs, outputs, or the compiled ignore set as unexpected (t.Error at line 1189). Add Ignore = [".databricks"] here.

Loading