Repository navigation
Conversation
`ssh connect` to serverless CPU compute now submits the SSH server job with environment version 5 when --environment-version is not set. GPU (--accelerator) stays on 4, because the standard version 5 and 6 environments ship no torch on GPU. An explicit --environment-version still wins, and base environments and dedicated clusters are unchanged. The single DefaultEnvironmentVersion constant becomes DefaultCPUEnvironmentVersion and DefaultGPUEnvironmentVersion, resolved from the accelerator by defaultEnvironmentVersion. The submit spec, telemetry and the default connection name all go through it, so the name hashes the version that is submitted. CPU default names change, moving CPU users to a new v5 server; GPU default names are byte-for-byte unchanged. The --environment-version flag now defaults to 0 (not set) instead of a single version, and its help names both defaults. That makes the post-parse reset to 0 unnecessary, and ToProxyCommand keeps persisting the flag only when it was set explicitly, so a GPU IDE connection still resolves to 4 through --accelerator. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
Automated review at 2c12b95. Each finding was verified against the code.
No blocking issues found. 1 non-blocking comment(s) below.
This PR splits the default serverless environment version by compute type: CPU defaults to 5, GPU stays on 4. The flag default moves to 0 (unset) so the actual default is resolved from --accelerator. Connection-name hashing, telemetry, and the submit path all correctly resolve through the new defaultEnvironmentVersion helper. The change is well-tested with pinned connection-name checks and a ProxyCommand round-trip test. One naming nit on the new test file.
| @@ -0,0 +1,47 @@ | |||
| package ssh | |||
There was a problem hiding this comment.
[nit] This file declares package ssh (not package ssh_test) to reach the unexported newConnectCommand. The repo convention for same-package tests is to name the file with an _internal_test.go suffix—see client_internal_test.go and submit_internal_test.go one directory over. Rename to connect_internal_test.go to match.
|
Commit: Regression test reportSelected 1 changed tests (limit=20, 0 not selected) 1 tests: regression=1 unreleased=0 coverage=0 failing=0
TestAccept/ssh/connect-serverless-cpu: branch ✅ | main (8b7e509) ❌ | latest (v1.27.0) ❌main (8b7e509): latest (v1.27.0): |
✅ Integration test reportCommit: 5bb9848
Top 6 slowest tests (at least 2 minutes):
|
Changes
databricks ssh connectto serverless CPU compute now submits the SSH server job with environment version 5 when--environment-versionis not set. Serverless GPU (--accelerator) stays on 4. An explicit--environment-versionstill wins, and--base-environmentand--clusterare unchanged.DefaultEnvironmentVersionis split intoDefaultCPUEnvironmentVersion(5) andDefaultGPUEnvironmentVersion(4). The submit spec, telemetry and the default connection name all resolve an unset version from--accelerator, so the name hashes the version that is actually submitted.--environment-version 4still maps to the old CPU name.--environment-versionnow defaults to 0 (not set), and its help names both defaults. The help example uses 6, since 5 is now the CPU default.Why
The tunnel stayed on version 4 because the REPL sandbox on CPU environments 5 and later blocked the driver proxy. The backend fix reached all production regions on 2026-11-05. Version 4 blocks IPv4 loopback inside the session, so a Jupyter kernel started over SSH never answers; on 5 it does.
GPU stays on 4 because the standard GPU environments 5 and 6 ship no PyTorch.
Tests
ssh connectflags.connect-serverless-cpusubmitting"5".connect-serverless-gpustill submits"4".--environment-version 5and6, on production workspaces in Azure westus, AWS us-west-2 and AWS us-east-1.%pipmagic registered, and Spark worked.This pull request and its description were written by Isaac.