Skip to content

Deploy the app when the deployment triggered by start failed - #7780

Merged
denik merged 2 commits into
mainfrom
denik/apps-failed-autodeploy
Nov 19, 2026
Merged

denik merged 2 commits into
mainfrom
denik/apps-failed-autodeploy

Conversation

@denik

@denik denik commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Changes

appdeploy.WaitForDeploymentToComplete no longer fails when the active or pending deployment it waits for ends in FAILED. It logs the failure and returns, and the caller runs the bundle's own deployment. Both callers (bundle run for apps and lifecycle.started in the direct engine) deploy right after waiting, so the new deployment replaces the failed one.

Why

Since #6399, Create sends source_code_path, so the app records a default_source_code_path. Starting a stopped app then makes the backend deploy that source with the app.yaml it contains, not the bundle's inline config. If that config doesn't run (in inline_config, app.yaml runs python app.py, which exits right away), the deployment fails with app exited unexpectedly. bundle run waited on that deployment, returned its error, and never deployed the inline config.

This is a regression from #6399. It is unreleased, so there is no changelog entry.

Decision: I fixed the wait instead of reverting source_code_path on Create, which #6399 added on purpose. A failed earlier deployment never needs to block the new one.

Tests

  • Unit tests in bundle/appdeploy: a FAILED deployment no longer returns an error. The test fails without the fix.
  • TestAccept/bundle/resources/apps/inline_config (both DMS variants) passes on AWS with the fix and fails without it. The run log shows Deployment ... failed (Error: app exited unexpectedly. ...), deploying again, followed by App started successfully.
  • The last commit asks the PR's cloud run to run inline_config on every cloud.

Dashboard

error/3769: 74 failures in 11 main runs on aws, azure and gcp, starting with the first run after #6399.

LOG.run_app: ✓ Waiting for the active deployment to complete...
LOG.run_app: Error: failed to reach SUCCEEDED, got FAILED: Error: app exited unexpectedly. Please check /logz for more details

This pull request and its description were written by Isaac.

Since #6399, Create sends source_code_path, so starting a stopped app
makes the backend deploy that source with its own app.yaml, without the
bundle's inline config. When that deployment fails, bundle run and
lifecycle.started returned its error instead of running the bundle's
deployment, which replaces it.

cloud: TestAccept/bundle/resources/apps/inline_config

Co-authored-by: Isaac <no-reply@databricks.com>
@denik denik added the ai-auto Deprecated: use factory-generated instead. label Nov 18, 2026
@github-actions github-actions Bot added the DABs DABs related issues label Nov 18, 2026
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 1b56ccf

Run: 44382736862

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 275 19 10:12
💚​ aws-windows-latest-4core-16gb 1 277 17 8:08
💚​ azure linux-2core-8gb 1 274 19 9:29
💚​ azure-windows-latest-4core-16gb 1 276 17 7:27
💚​ gcp linux-2core-8gb 1 275 19 11:19
💚​ gcp-windows-latest-4core-16gb 1 277 17 7:14
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 21 slowest tests (at least 2 minutes):
duration env testname
4:12 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/inline_config/DMS=
4:07 azure linux-2core-8gb TestAccept/bundle/resources/apps/inline_config/DMS=true
3:42 azure linux-2core-8gb TestAccept/bundle/resources/apps/inline_config/DMS=
3:19 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/inline_config/DMS=true
3:14 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/inline_config/DMS=
3:11 aws-windows-latest-4core-16gb TestAccept
3:06 azure-windows-latest-4core-16gb TestAccept
3:05 aws linux-2core-8gb TestAccept/bundle/resources/apps/inline_config/DMS=true
3:01 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/inline_config/DMS=true
2:52 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/inline_config/DMS=true
2:50 gcp linux-2core-8gb TestAccept/bundle/resources/apps/inline_config/DMS=
2:46 aws linux-2core-8gb TestAccept/bundle/resources/apps/inline_config/DMS=
2:40 gcp-windows-latest-4core-16gb TestAccept
2:39 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/inline_config/DMS=
2:25 aws linux-2core-8gb TestFilerRecursiveDelete/workspace_files_extensions
2:22 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:11 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsStat
2:10 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsStat
2:05 gcp linux-2core-8gb TestAccept/bundle/resources/apps/inline_config/DMS=true
2:04 aws-windows-latest-4core-16gb TestWorkspaceFilesExtensions_ExportFormatIsPreserved/source_sql
2:01 gcp-windows-latest-4core-16gb TestImportDir

@denik
denik marked this pull request as ready for review November 18, 2026 18:26
@denik
denik requested review from a team as code owners November 18, 2026 18:26

@andrewnester andrewnester left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings from a full pass; I'll take a manual look at the appdeploy changes before approving.

@Sankalp-Mittal Sankalp-Mittal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Generated] Review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.

Comment thread bundle/appdeploy/app.go Resolved
Comment thread bundle/appdeploy/app_test.go Resolved
…turned

cloud: TestAccept/bundle/resources/apps/inline_config

Co-authored-by: Isaac <no-reply@databricks.com>

@andrewnester andrewnester left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking findings from a full pass; I'll take a manual look at the appdeploy changes before approving.

@Sankalp-Mittal Sankalp-Mittal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI Generated] Follow-up review by review-factory (Claude Fable 5.2 + Claude Opus 5.5, cross-verified). Only points the reviewers agreed on are posted.

No new issues found.

@denik
denik added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit a4661b0 Nov 19, 2026
27 checks passed
@denik
denik deleted the denik/apps-failed-autodeploy branch November 19, 2026 13:33
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: a4661b0

Run: 44534815075

Env 💥​PANIC ❌​FAIL 🟨​KNOWN 🤯​MISS 💚​RECOVERED ✅​pass 🙈​skip Time
💥​ aws linux-2core-8gb 1 2 727 15 885 1392 320:00
💥​ aws-windows-latest-4core-16gb 1 2 2 641 18 882 1420 320:04
💥​ azure linux-2core-8gb 1 6 930 9 664 1400 320:00
💥​ azure-windows-latest-4core-16gb 1 6 630 14 875 1428 320:03
💥​ gcp linux-2core-8gb 1 776 15 825 1398 320:00
💥​ gcp-windows-latest-4core-16gb 1 606 20 906 1426 320:03
1259 interesting tests: 1233 MISS, 9 RECOVERED, 8 KNOWN, 5 PANIC, 4 FAIL
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🤯​ TestAccept 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/apps/job_permissions/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_doesnot_exist/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/artifacts/artifact_path_with_volume/volume_not_deployed/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cli_defaults/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_permissions/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/config_edits/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/dashboard_etag/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/external_cluster_policy_cross_target/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/flushed_cache/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/formatting_preserved/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_multiple_tasks/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_params_variables/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/job_pipeline_task/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_missing_state/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/json_status_selectors/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_files/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_json/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_json/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/output_no_changes/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/pipeline_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/resolve_variables/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_basic/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/select_multiple/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_deployment/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/skip_permissions/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/target_override/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/target_override/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert 🤯​M 🙈​s 🤯​M 🙈​s 🤯​M 🙈​s
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DMS= 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/config-remote-sync/task_rename_revert/DMS=true 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/debug/fetch-repository-info/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/empty-bundle/DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_YAML_SYNC=true/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/files/no-snapshot-sync/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/basic 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/basic/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/breakglass 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/breakglass/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/can-manage 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/can-manage/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/destroy-redeploy 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/destroy-redeploy/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/dev-mode 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/dev-mode/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/legacy-state 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/legacy-state/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/missing-snapshot 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/missing-snapshot/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/no-artifacts/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/permissions-change 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/permissions-change/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/select 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/immutable/select/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/mlops-stacks/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deploy/spark-jar-task/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/alert/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/catalog/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster/DMS= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/cluster/DMS=true 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=true/READPLAN= 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
🤯​ TestAccept/bundle/deployment/bind/dashboard/DMS=true/READPLAN=1 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M 🤯​M
Top 50 slowest tests (at least 2 minutes):
duration env testname
15:26 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
15:25 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
14:08 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
14:00 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
13:14 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
12:50 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
12:28 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
12:09 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
12:06 azure-windows-latest-4core-16gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=
12:02 azure-windows-latest-4core-16gb TestAccept/bundle/resources/schemas/rename_then_change/DMS=true
11:47 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
11:28 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
11:10 azure-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
11:03 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true
11:01 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:57 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
10:47 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
10:39 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
10:33 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:57 aws linux-2core-8gb TestAccept/bundle/quiet-levels/DMS=
9:54 azure linux-2core-8gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=pipeline_apply_policy_default_values.yml.tmpl/READPLAN=1
9:48 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
9:44 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:43 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true
9:35 azure linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
9:28 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
9:27 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
9:17 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/materialization_sets/batch/DMS=true
9:05 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/jobs/fail-on-active-runs/DMS=true
9:00 gcp-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=yes/DMS=/NBOOK=yes/PY=yes/READPLAN=1
9:00 azure linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=genie_space.yml.tmpl/READPLAN=
8:55 azure linux-2core-8gb TestAccept/bundle/quiet-levels/DMS=true
8:50 azure linux-2core-8gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=model_with_permissions.yml.tmpl/READPLAN=
8:45 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
8:44 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/materialization_sets/immutable/DMS=true
8:41 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:35 gcp linux-2core-8gb TestAccept/bundle/resources/volumes/comment_out_of_band/DMS=true
8:29 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=
8:27 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=true
8:24 azure linux-2core-8gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=model.yml.tmpl/READPLAN=
8:20 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
8:14 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
8:14 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
8:13 gcp linux-2core-8gb TestAccept/bundle/quiet-levels/DMS=
8:13 aws-windows-latest-4core-16gb TestAccept/bundle/templates/default-python/combinations/serverless/DLT=yes/DMS=/NBOOK=yes/PY=yes/READPLAN=1
8:07 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started-omitted/DMS=
8:03 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
8:02 azure-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=volume.yml.tmpl/READPLAN=
8:00 aws-windows-latest-4core-16gb TestAccept/bundle/resources/registered_models/basic/DMS=true
7:57 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/materialization_sets/batch/DMS=
(1113 table rows omitted to keep the report under 60000 bytes)

@denik denik added factory-generated and removed ai-auto Deprecated: use factory-generated instead. labels Nov 20, 2026
idelafuente-oc pushed a commit to idelafuente-oc/cli that referenced this pull request Nov 20, 2026
…cks#7804)

## Changes

Fix a race in the interactive list pager (`libs/cmdio/pager.go`) that
could drop or reorder a page of rows, and the macOS flake it caused in
`TestPagedTemplateRespectsLimit`.

A non-final page was printed by returning a bare `tea.Println` cmd.
bubbletea runs every returned cmd on its own goroutine, so nothing
ordered that print before later messages. When ENTER (or SPACE) started
the next fetch and that page was the last one, its `tea.Sequence(print,
tea.Quit)` could reach the event loop first, and the program quit before
the earlier page was printed. In the test (page size 5, limit 7) that
leaves only the last 2 rows: `expected: 7, actual: 2`.

A non-final page is now followed by a `pagePrintedMsg` in the same
`tea.Sequence` as its print. `tea.Sequence` delivers each message before
running the next cmd, and `p.Send` is unbuffered, so when
`pagePrintedMsg` arrives the page's lines are already queued. Until then
a new `printing` flag holds keys back: SPACE is ignored, ENTER sets
`drainAll`, and `q`/`esc`/Ctrl+C set `quitPending`. The `pagePrintedMsg`
handler then quits or chains the next fetch, replacing the old
`Sequence(print, fetch)` branch. `View` is unchanged: the prompt shows
as soon as the batch arrives.

Also in the tests: `countContentLines` and
`TestPagedTemplateMatchesNonPagedForSmallList` now read the output the
way a terminal shows it, keeping only the text after each line's last
`\r`. tea redraws its spinner and prompt in place, so plain ANSI
stripping left that text glued to real rows (`"\r⣽ loading...\r1 1"`).
`TestPagedTemplateMatchesNonPagedForSmallList` failed once that way, and
`countContentLines` would skip a row that lands on the prompt's line.

Introduced with the pager in databricks#5015.

## Why

Users can lose rows of `list` output in the pager. It is rare at human
typing speed, but it fails CI on unrelated PRs (macOS runners).

## Tests

- New `TestPagerModelKeysWaitForPagePrint` and
`TestPagerModelQuitWaitsForPagePrint` fail without the fix (SPACE/ENTER
fetch, q quits, before the page is printed).
`TestPagedTemplateDrainsFullIterator` now checks row order, not just
presence. Updated `TestPagerModelBatchPromptsWhenMore` and
`TestPagerModelBatchDrainingChainsFetch` to the new message flow.
- Reproduced the flake deterministically by adding a 20ms sleep to the
page print cmd: before the fix, `TestPagedTemplateDrainsFullIterator`
got 18 of 23 rows and `TestPagedTemplateRespectsLimit` 2 of 7. With the
fix, every `TestPagedTemplate*` passes with the sleep in place.
- `go test -race ./libs/cmdio/ -count=200`, `go test ./libs/cmdio/
-count=1000` and `GOMAXPROCS=1 ... -count=200` pass.

## Dashboard

| error / test | failure |
|---|---|
|
[error/3590](http://go/cli/dashboard/error/3590?main=1&pr=1&end=2026-11-19T12:00:00Z&last=7d)
| `paged_template_test.go:90: Not equal: expected: 7 actual : 2` (3
failures in 3 days, macOS, PRs databricks#4738, databricks#7552, databricks#7780) |
|
[libs/cmdio.TestPagedTemplateRespectsLimit](http://go/cli/dashboard/test/libs/cmdio.TestPagedTemplateRespectsLimit?main=1&pr=1&end=2026-11-19T12:00:00Z&last=7d)
| same |
|
[libs/cmdio.TestPagedTemplateMatchesNonPagedForSmallList](http://go/cli/dashboard/test/libs/cmdio.TestPagedTemplateMatchesNonPagedForSmallList?main=1&pr=1&end=2026-11-19T12:00:00Z&last=7d)
| `expected: "1 1" actual : "\r⣽ loading...\r1 1"` (1 failure,
2026-11-16, Windows) |

This pull request and its description were written by Isaac.

---------

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues factory-generated

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants