Skip to content

air: Disable MLflow artifact log fallback by default - #7697

Draft
ethanma-db wants to merge 2 commits into
mainfrom
ethanma-db/air-bricklens-logs-default
Draft

ethanma-db wants to merge 2 commits into
mainfrom
ethanma-db/air-bricklens-logs-default

Conversation

@ethanma-db

@ethanma-db ethanma-db commented Nov 16, 2026 •

Copy link
Copy Markdown

Changes

Use Bricklens for AIR log streaming and downloads by default. Empty streams no longer probe MLflow, and unavailable Bricklens logs surface an error instead of silently switching backends.

Temporarily restore the legacy artifact fallback and artifact downloads with DATABRICKS_AIR_ENABLE_MLFLOW_LOG_FALLBACK=true. The default download path pages all node logs without the streaming tail cap and preserves partial files on fetch failures.

Why

The MLflow artifact ingestion path is being removed now that Bricklens logging is standard. Retain an environment-variable workaround during the transition, as agreed in the Slack discussion.

Related: #7621.

Tests

  • Focused tests and race checks cover opt-in fallback, empty active/static streams, paginated multi-node downloads, partial failures, and cancellation.
  • golangci-lint run ./cmd/air/...: 0 issues.
  • tools/validate_nextchanges.py: passed.
  • Full AIR suite passed: env -u DATABRICKS_CONFIG_PROFILE DATABRICKS_CONFIG_FILE=/dev/null go test ./cmd/air/... (161 s). The isolated config prevents the developer profile from supplying a workspace ID to URL tests.

@github-actions github-actions Bot added the AIR Databricks AI Runtime CLI label Nov 16, 2026
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 34969f2

Run: 43933268751

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
🔄​ aws linux-2core-8gb 1 1 269 15 11:21
💚​ aws-windows-latest-4core-16gb 1 272 13 8:00
💚​ azure linux-2core-8gb 1 269 15 13:23
💚​ azure-windows-latest-4core-16gb 1 271 13 8:26
💚​ gcp linux-2core-8gb 1 270 15 9:37
💚​ gcp-windows-latest-4core-16gb 1 272 13 7:42
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestSyncIncrementalSyncPythonNotebookToFile 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
Top 16 slowest tests (at least 2 minutes):
duration env testname
4:31 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
4:22 gcp-windows-latest-4core-16gb TestAccept
4:02 azure-windows-latest-4core-16gb TestAccept
3:53 aws-windows-latest-4core-16gb TestAccept
3:14 azure linux-2core-8gb TestFilerRecursiveDelete/workspace_files
2:40 azure-windows-latest-4core-16gb TestFilerRecursiveDelete/workspace_files
2:37 azure linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:27 azure-windows-latest-4core-16gb TestSyncFullFileSync
2:23 aws linux-2core-8gb TestFilerWorkspaceFilesExtensionsReadDir
2:23 azure-windows-latest-4core-16gb TestFilerWorkspaceFilesExtensionsDelete
2:23 aws linux-2core-8gb TestImportDirWithOverwriteFlag
2:11 azure linux-2core-8gb TestExportDir
2:09 aws linux-2core-8gb TestFilerReadDir/workspace_files
2:07 gcp-windows-latest-4core-16gb TestExportDir
2:01 azure-windows-latest-4core-16gb TestImportDirWithOverwriteFlag
2:00 azure linux-2core-8gb TestFilerReadWrite/workspace_files

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIR Databricks AI Runtime CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants