Skip to content

acc: show what starting a bundle-created app does to its deployments - #7809

Draft
janniklasrose wants to merge 1 commit into
mainfrom
janniklasrose/apps-start-deployment-tests
Draft

janniklasrose wants to merge 1 commit into
mainfrom
janniklasrose/apps-start-deployment-tests

Conversation

@janniklasrose

Copy link
Copy Markdown
Member

Changes

Adds four cloud acceptance tests under acceptance/bundle/resources/apps/start-triggered-deployment. Each creates an app with source_code_path and an inline config, starts its compute, and prints the deployments on the app:

  • apps-start: bundle deploy, then a plain apps start. The source folder has an app.yaml that runs, with a different env var than the inline config.
  • bundle-run: bundle deploy, then bundle run. No app.yaml in the source folder.
  • lifecycle-started-on-create: lifecycle.started: true from the first deploy. No app.yaml.
  • lifecycle-started-on-update: created with started: false, then set to true. No app.yaml.

Why

Since #6399 the Create request records source_code_path on the app, and the nightly inline_config test fails with app exited unexpectedly while bundle run waits on a deployment it did not make (see #7780). These tests are meant to show on a real workspace which commands are affected and what the app ends up with.

This is a diagnostic draft, not meant to merge as is. The goldens are generated against the local test server, which does not deploy on start, so they hold what the bundle asks for: the only deployment is the one the CLI makes, with the inline config. I expect the cloud run to fail, and the diffs are the result.

Tests

The four tests pass locally. The last commit message asks the cloud run to run them on every cloud.

This pull request and its description were written by Isaac.

Four cloud tests for apps created with source_code_path (#6399): a plain apps start, bundle run, and lifecycle.started on create and on update. The goldens hold what the bundle asks for, so a cloud diff shows a deployment the bundle did not make or a command that fails because of one.

cloud: TestAccept/bundle/resources/apps/start-triggered-deployment

Co-authored-by: Isaac <no-reply@databricks.com>
@github-actions github-actions Bot added DABs DABs related issues test-only-fix PRs that update failing of flaky tests without changing CLI code. labels Nov 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Commit: 173b8beb5 | Workflow run

Regression test report

Selected 4 changed tests (limit=20, 0 not selected)

4 tests: regression=0 unreleased=0 coverage=4 failing=0
test branch main (4f2fda3) latest (v1.27.0)
TestAccept/bundle/resources/apps/start-triggered-deployment/apps-start ✅ ✅ ✅
TestAccept/bundle/resources/apps/start-triggered-deployment/bundle-run ✅ ✅ ✅
TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-create ✅ ✅ ✅
TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-update ✅ ✅ ✅

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

❌ Integration test report

Commit: 173b8be

Run: 44543623670

Env ❌​FAIL 🟨​KNOWN 🔄​flaky ✅​pass 🙈​skip Time
❌​ aws linux-2core-8gb 3 1 273 19 43:05
❌​ aws-windows-latest-4core-16gb 3 1 275 17 36:32
❌​ azure linux-2core-8gb 2 1 1 272 19 41:07
❌​ azure-windows-latest-4core-16gb 3 1 274 17 38:32
❌​ gcp linux-2core-8gb 1 1 275 19 28:18
❌​ gcp-windows-latest-4core-16gb 2 1 2 274 17 41:09
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K
❌​ TestAccept/bundle/resources/apps/start-triggered-deployment/apps-start ❌​F ❌​F ❌​F ❌​F ❌​F ❌​F
❌​ TestAccept/bundle/resources/apps/start-triggered-deployment/bundle-run ❌​F ❌​F 🔄​f ❌​F ✅​p ❌​F
🔄​ TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-create ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
❌​ TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-update ❌​F ❌​F ❌​F ❌​F ✅​p 🔄​f
Top 37 slowest tests (at least 2 minutes):
duration env testname
8:47 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-update
4:56 gcp linux-2core-8gb TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-update
4:50 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsStat
4:45 azure linux-2core-8gb TestAccept/bundle/resources/apps/start-triggered-deployment/bundle-run
4:41 gcp linux-2core-8gb TestAccept/bundle/resources/apps/start-triggered-deployment/bundle-run
4:12 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsDelete
3:57 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-create
3:45 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
3:36 aws linux-2core-8gb TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-create
3:34 azure linux-2core-8gb TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-create
3:33 gcp linux-2core-8gb TestFilerReadWrite/workspace_files_extensions
3:26 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-create
3:19 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
3:13 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
3:12 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsStat
3:10 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
3:08 aws linux-2core-8gb TestImportDirWithOverwriteFlag
3:01 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-create
3:00 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:51 aws linux-2core-8gb TestFilerRecursiveDelete/workspace_files
2:49 azure-windows-latest-4core-16gb TestLock
2:47 gcp linux-2core-8gb TestWorkspaceFilesExtensions_ExportFormatIsPreserved/source_r
2:43 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsDelete
2:34 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:31 aws-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files_extensions
2:24 gcp linux-2core-8gb TestAccept/bundle/resources/apps/start-triggered-deployment/lifecycle-started-on-create
2:24 aws-windows-latest-4core-16gb TestLock
2:20 azure-windows-latest-4core-16gb TestImportDirWithOverwriteFlag
2:18 aws-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsReadDir
2:17 azure linux-2core-8gb TestFilerRecursiveDelete/workspace_files
2:13 gcp-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files
2:12 gcp-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsRead
2:12 azure-windows-latest-4core-16gb TestExportDir
2:06 gcp-windows-latest-4core-16gb TestFilerReadWrite/workspace_files_extensions
2:04 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsStat
2:02 azure linux-2core-8gb TestSyncFullFileSync
2:00 azure-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files_extensions

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues test-only-fix PRs that update failing of flaky tests without changing CLI code.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants