From d5a6fdf7cbb535be85b80da5498830714c30a138 Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 17:25:16 +0000 Subject: [PATCH 1/7] feat(kernel): honor WithPort on the kernel backend The kernel backend rejected any WithPort other than 443. The kernel has no port setter, but it keeps a `host:port` authority in its host, so the connector now folds a non-default port into the host it passes to the kernel. - kernelHostWithPort adds a non-default port with net.JoinHostPort (IPv6-safe). The default 443 and an unset port add nothing. - validateKernelHostPort rejects an out-of-range port, a host with unbalanced IPv6 brackets (for example `[::1` or `h.databricks.com]`), and a supplied port that disagrees with a port embedded in the host. An unset port (WithDefaults fills 443) defers to the embedded port. An explicit WithPort(443) or DSN port is supplied, so it must match an embedded port too (AC-P3: never drop a port). - UserConfig gains an internal PortExplicit flag, set by WithPort and the DSN parser. It is not a public option: internal/ is not importable by users. - Non-https protocols and the other kernel rejections are unchanged. With OAuth (M2M and U2M) and a non-default port, the kernel reaches the OAuth endpoints at the same host:port as the connection. Thrift keeps OAuth on the bare host. This is documented in CONNECTION_PARAMETERS.md, the README and the CHANGELOG, and is not guarded. Tests: one table-driven TestKernelCustomPort covers the host mapping, the validation (accept and reject, including explicit 443 and unbalanced brackets), and the kernel Host. The config, connector and driver tests expect PortExplicit where WithPort or the DSN set the port. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 + CONNECTION_PARAMETERS.md | 2 +- README.md | 2 +- connector.go | 1 + connector_test.go | 1 + driver_test.go | 1 + internal/config/config.go | 11 +++-- internal/config/config_test.go | 25 ++++++++++ kernel_config.go | 66 +++++++++++++++++++++---- kernel_config_test.go | 90 ++++++++++++++++++++++++++++++++-- 10 files changed, 182 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4d92df40..29847d14 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,8 @@ ## Unreleased (v2.0.0 prototype) +- Honor `WithPort` / DSN port on the kernel backend: a non-default port is folded into the host (`host:port`, IPv6-safe) instead of being rejected, and a port that conflicts with one embedded in the host, or is out of range, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). + - Fix the module-root package conflict when building, linting, or running Gazelle with the `tools` build tag (databricks/databricks-sql-go#162). - **Behavior change:** Report declared DECIMAL precision and scale through `sql.ColumnType.DecimalSize()` on Thrift and SEA, including zero-row results. Tools such as ORM migrators can now act on these qualifiers (databricks/databricks-sql-go#500). - **Behavior change:** Support Arrow TIME results on Thrift and SEA as UTC `time.Time` values on `0001-01-01`, preserving precision and using clock-only nested JSON. TIME metadata now reports `TIME`/`time.Time` with no length, including empty and NULL-only results; previously Thrift reported `STRING`/`string` with unbounded length, and SEA reported an empty name with `*interface{}` (databricks/databricks-sql-go#505). diff --git a/CONNECTION_PARAMETERS.md b/CONNECTION_PARAMETERS.md index 28624b37..7d06a065 100644 --- a/CONNECTION_PARAMETERS.md +++ b/CONNECTION_PARAMETERS.md @@ -40,7 +40,7 @@ wire. They are listed under [Kernel client-result confs](#kernel-client-result-c |---|---|:---:|:---:|---|---| | *(host)* | `WithServerHostname` | ✅ | ✅ | *(required)* | Workspace hostname. | | *(path)* | `WithHTTPPath` | ✅ | ✅ | required on Thrift except localhost; required on kernel unless `warehouseId` is set | Warehouse/endpoint HTTP path. On the kernel path, a bare warehouse id can be used instead. | -| *(port)* | `WithPort` | ✅ | ⚠️ | `443` for connector options; required in a DSN | Kernel accepts **443 only** and rejects any other port. | +| *(port)* | `WithPort` | ✅ | ✅ | `443` for connector options; required in a DSN | Kernel folds a non-default port into the host it connects to; an out-of-range or conflicting port (including an explicit `WithPort(443)` against a host port) is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | | *(scheme)* | *(via `WithServerHostname`)* | ✅ | ⚠️ | `https` | Kernel accepts `https` only and rejects `http`. A bare `localhost` passed to `WithServerHostname` selects `http`; a scheme-less DSN, including localhost, is prefixed with `https`. | | `warehouseId` | `WithWarehouseID` | ⚠️ | ✅ | | Bare warehouse id. The kernel normally prefers it over the HTTP path; a canonical path carrying `?o=` takes precedence so workspace-org routing is preserved. The Thrift backend **silently ignores** it. | | `catalog` | `WithInitialNamespace` | ✅ | ✅ | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. | diff --git a/README.md b/README.md index c6b98b85..8f784bfd 100644 --- a/README.md +++ b/README.md @@ -240,7 +240,7 @@ session parameter on both backends. |---|---|---|---|---| | *(host)* | `WithServerHostname` | Both | *(required)* | Workspace hostname. | | *(path)* | `WithHTTPPath` | Both | *(required)* | Warehouse/endpoint HTTP path. | -| *(port)* | `WithPort` | Thrift only | `443` | Kernel connects on **443 only** and rejects any other port. | +| *(port)* | `WithPort` | Both | `443` | Kernel folds a non-default port into the host it connects to; an out-of-range or conflicting port is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | | `warehouseId` | `WithWarehouseID` | SEA only | | Bare warehouse id; the kernel routes by it (preferred over the HTTP path). **The Thrift backend ignores it.** | | `catalog` | `WithInitialNamespace` | Both | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. | | `schema` | `WithInitialNamespace` | Both | | Initial schema. Kernel applies it post-connect via `USE SCHEMA`. | diff --git a/connector.go b/connector.go index 1b72c619..34c1350d 100644 --- a/connector.go +++ b/connector.go @@ -367,6 +367,7 @@ func parseHostName(host string) (protocol, hostname string) { func WithPort(port int) ConnOption { return func(c *config.Config) { c.Port = port + c.PortExplicit = true } } diff --git a/connector_test.go b/connector_test.go index 684a8646..2ecb0266 100644 --- a/connector_test.go +++ b/connector_test.go @@ -174,6 +174,7 @@ func TestNewConnector(t *testing.T) { expectedUserConfig := config.UserConfig{ Host: host, Port: port, + PortExplicit: true, Protocol: "https", AccessToken: accessToken, Authenticator: &pat.PATAuth{AccessToken: accessToken}, diff --git a/driver_test.go b/driver_test.go index 53aba0eb..5aa6f4e8 100644 --- a/driver_test.go +++ b/driver_test.go @@ -21,6 +21,7 @@ func TestOOpenConnector(t *testing.T) { expectedUserConfig := config.UserConfig{ Host: host, Port: port, + PortExplicit: true, Protocol: "https", AccessToken: accessToken, HTTPPath: "/" + httpPath, diff --git a/internal/config/config.go b/internal/config/config.go index f9c5143e..3d808970 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -265,9 +265,12 @@ func (c *Config) DeepCopy() *Config { // UserConfig is the set of configurations exposed to users type UserConfig struct { - Protocol string - Host string // from databricks UI - Port int // from databricks UI + Protocol string + Host string // from databricks UI + Port int // from databricks UI + // PortExplicit is true once WithPort or the DSN supplied Port (WithDefaults fills 443 otherwise). + // Internal only: kernel validation uses it to reject an explicit port that conflicts with a host port. + PortExplicit bool HTTPPath string // from databricks UI Catalog string Schema string @@ -342,6 +345,7 @@ func (ucfg UserConfig) DeepCopy() UserConfig { Protocol: ucfg.Protocol, Host: ucfg.Host, Port: ucfg.Port, + PortExplicit: ucfg.PortExplicit, HTTPPath: ucfg.HTTPPath, Catalog: ucfg.Catalog, Schema: ucfg.Schema, @@ -455,6 +459,7 @@ func ParseDSN(dsn string) (UserConfig, error) { return UserConfig{}, dbsqlerrint.NewRequestError(context.TODO(), dbsqlerr.ErrInvalidDSNPort, err) } ucfg.Port = port + ucfg.PortExplicit = true ucfg.HTTPPath = parsedURL.Path diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 518ccaf2..07c1bf7f 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -38,6 +38,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, + PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", @@ -58,6 +59,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, + PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -78,6 +80,7 @@ func TestParseConfig(t *testing.T) { Protocol: "http", Host: "localhost", Port: 8080, + PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &noop.NoopAuth{}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -97,6 +100,7 @@ func TestParseConfig(t *testing.T) { Protocol: "http", Host: "localhost", Port: 8080, + PortExplicit: true, Authenticator: &noop.NoopAuth{}, MaxRows: defaultMaxRows, SessionParams: make(map[string]string), @@ -115,6 +119,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -136,6 +141,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -159,6 +165,7 @@ func TestParseConfig(t *testing.T) { Host: "example.cloud.databricks.com", Authenticator: &noop.NoopAuth{}, Port: 8000, + PortExplicit: true, MaxRows: defaultMaxRows, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", SessionParams: make(map[string]string), @@ -177,6 +184,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -198,6 +206,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -219,6 +228,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, @@ -240,6 +250,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -265,6 +276,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -286,6 +298,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -312,6 +325,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -342,6 +356,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, + PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -362,6 +377,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, + PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, QueryTimeout: 100 * time.Second, @@ -419,6 +435,7 @@ func TestParseConfig(t *testing.T) { args: args{dsn: "token:supersecret2@:443?catalog=default&schema=system&timeout=100&maxRows=1000"}, wantCfg: UserConfig{ Port: 443, + PortExplicit: true, Protocol: "https", AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, @@ -443,6 +460,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -467,6 +485,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -491,6 +510,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -545,6 +565,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, + PortExplicit: true, Authenticator: m2m.NewAuthenticator("client_id", "client_secret", "example.cloud.databricks.com"), HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", QueryTimeout: 100 * time.Second, @@ -571,6 +592,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, + PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", @@ -597,6 +619,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, + PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", @@ -625,6 +648,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, + PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", @@ -646,6 +670,7 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, + PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", diff --git a/kernel_config.go b/kernel_config.go index 21a9f3cb..c6c3a4dd 100644 --- a/kernel_config.go +++ b/kernel_config.go @@ -4,6 +4,7 @@ import ( "context" "errors" "fmt" + "net" "net/url" "strconv" "strings" @@ -73,18 +74,14 @@ func validateKernelConfigContext(ctx context.Context, cfg *config.Config) (kerne // folds its server conf (spark.sql.thriftserver.metadata.metricview.enabled=true) // into SessionConf backend-neutrally, so the kernel path sends the identical conf // the Thrift path does. No per-backend handling needed here. - // Port / Protocol: the kernel C ABI takes only a bare host and connects over - // https:443; it has no port or scheme setter. The Thrift path honors a custom - // port/scheme via ToEndpointURL, so a non-default value here would be silently - // ignored on the kernel path (it would just hit 443) — reject it instead, per - // the "nothing silently ignored" contract. Defaults (https/443) are fine. + // Protocol: the kernel always connects over https, so a non-https value would be ignored. if cfg.Protocol != "" && cfg.Protocol != "https" { return kernel.Auth{}, fmt.Errorf("databricks: a non-https protocol is %w "+ "(it connects over https); use the default (Thrift) backend", dbsqlerr.ErrNotSupportedByKernel) } - if cfg.Port != 0 && cfg.Port != 443 { - return kernel.Auth{}, fmt.Errorf("databricks: a non-default port (WithPort) is %w "+ - "(it connects on 443); omit it or use the default (Thrift) backend", dbsqlerr.ErrNotSupportedByKernel) + // Port: the kernel folds host:port (see kernelHostWithPort); reject a bad port or a conflict here. + if err := validateKernelHostPort(cfg.Host, cfg.Port, cfg.PortExplicit); err != nil { + return kernel.Auth{}, err } // Transport (WithTransport, a custom http.RoundTripper carrying a custom CA // bundle / mTLS / proxy): the kernel uses its own Rust HTTP stack below the C @@ -155,7 +152,7 @@ func validateKernelConfigContext(ctx context.Context, cfg *config.Config) (kerne // with newKernelBackend's kernel.Config assembly. func buildKernelConfig(cfg *config.Config, kauth kernel.Auth) kernel.Config { kc := kernel.Config{ - Host: cfg.Host, + Host: kernelHostWithPort(cfg.Host, cfg.Port), HTTPPath: cfg.HTTPPath, WarehouseID: cfg.WarehouseID, Auth: kauth, @@ -239,6 +236,57 @@ func buildKernelConfig(cfg *config.Config, kauth kernel.Auth) kernel.Config { return kc } +// unbalancedBrackets reports a host whose IPv6 brackets do not pair up: an unclosed `[`, text +// after the closing `]` other than a `:port`, or a stray bracket in a plain name. +func unbalancedBrackets(host string) bool { + if !strings.HasPrefix(host, "[") { + return strings.ContainsAny(host, "[]") + } + end := strings.IndexByte(host, ']') + return end < 0 || (end+1 < len(host) && host[end+1] != ':') +} + +// validateKernelHostPort rejects an out-of-range port, a host with unbalanced IPv6 brackets, and a +// supplied port that disagrees with a port embedded in the host. explicit is true when WithPort or +// the DSN set the port. WithDefaults fills 443 otherwise, so an unset 443 defers to the embedded port. +func validateKernelHostPort(host string, port int, explicit bool) error { + if port < 0 || port > 65535 { + return fmt.Errorf("databricks: WithPort %d is out of range (want 1..65535): %w", port, dbsqlerr.ErrInvalidKernelConfig) + } + if unbalancedBrackets(host) { + return fmt.Errorf("databricks: host %q has unbalanced IPv6 brackets: %w", host, dbsqlerr.ErrInvalidKernelConfig) + } + _, p, err := net.SplitHostPort(host) + if err != nil { + return nil // no embedded port + } + embedded, err := strconv.Atoi(p) + if err != nil || embedded < 1 || embedded > 65535 { + return fmt.Errorf("databricks: host %q carries an invalid port %q (want 1..65535): %w", host, p, dbsqlerr.ErrInvalidKernelConfig) + } + if supplied := port != 0 && (explicit || port != 443); supplied && port != embedded { + return fmt.Errorf("databricks: host %q carries port %d but WithPort is %d: %w", host, embedded, port, dbsqlerr.ErrInvalidKernelConfig) + } + return nil +} + +// kernelHostWithPort returns the Host the kernel connects to. The kernel has no port setter but +// keeps a `host:port` authority, so a non-default port is folded in (IPv6-safe). 0 and 443 add +// nothing, and a host that already carries a port is returned unchanged. +func kernelHostWithPort(host string, port int) string { + if port == 0 || port == 443 { + return host + } + if _, _, err := net.SplitHostPort(host); err == nil { + return host + } + name := host + if strings.HasPrefix(host, "[") && strings.HasSuffix(host, "]") { + name = host[1 : len(host)-1] + } + return net.JoinHostPort(name, strconv.Itoa(port)) +} + func kernelTelemetryConfig(cfg *config.Config) *kernel.TelemetryConfig { var enabled *bool if val, isSet := cfg.EnableTelemetry.Get(); isSet { diff --git a/kernel_config_test.go b/kernel_config_test.go index d8cd3c0c..30282689 100644 --- a/kernel_config_test.go +++ b/kernel_config_test.go @@ -121,7 +121,6 @@ func TestValidateKernelConfig(t *testing.T) { }{ {"query timeout", func(c *config.Config) { c.QueryTimeout = 30 * time.Second }}, {"non-https protocol", func(c *config.Config) { c.Protocol = "http" }}, - {"non-default port", func(c *config.Config) { c.Port = 8443 }}, {"custom transport", func(c *config.Config) { c.Transport = http.DefaultTransport }}, } for _, tc := range rejections { @@ -443,10 +442,11 @@ var kernelConfigFieldDisposition = map[string]string{ "Schema": "forwarded", // Rejected loudly by validateKernelConfig. - "QueryTimeout": "rejected", // when > 0 (WithTimeout) - "Protocol": "rejected", // kernel is https-only; non-default rejected - "Port": "rejected", // kernel connects on 443; non-default rejected - "Transport": "rejected", // custom RoundTripper; kernel uses its own HTTP stack, so reject rather than drop + "QueryTimeout": "rejected", // when > 0 (WithTimeout) + "Protocol": "rejected", // kernel is https-only; non-default rejected + "Port": "forwarded", // folded into the kernel Host by kernelHostWithPort; validated by validateKernelHostPort + "PortExplicit": "forwarded", // set by WithPort / the DSN; consumed by validation, not sent to the kernel + "Transport": "rejected", // custom RoundTripper; kernel uses its own HTTP stack, so reject rather than drop // Forwarded to the kernel's HTTP retry config via kernelRetryConfig → // set_retry_config (WithRetries: backoff bounds + max attempts, incl. disable). @@ -972,3 +972,83 @@ func TestKernelRetryConfig(t *testing.T) { } }) } + +// TestKernelCustomPort covers WithPort on the kernel backend (AC-P1, AC-P3): a non-default port is +// folded into the kernel Host, 0 and 443 add nothing, and an out-of-range port, unbalanced IPv6 +// brackets, or a supplied port that disagrees with an embedded port is rejected with +// ErrInvalidKernelConfig. +func TestKernelCustomPort(t *testing.T) { + cases := []struct { + name string + host string + port int + explicit bool + wantHost string // kernelHostWithPort result for an accepted pair + wantErr bool + }{ + {"default 443 adds nothing", "h.databricks.com", 443, false, "h.databricks.com", false}, + {"unset port adds nothing", "h.databricks.com", 0, false, "h.databricks.com", false}, + {"custom port", "h.databricks.com", 8443, false, "h.databricks.com:8443", false}, + {"IPv4 literal", "127.0.0.1", 8443, false, "127.0.0.1:8443", false}, + {"bare IPv6 is bracketed", "::1", 8443, false, "[::1]:8443", false}, + {"bracketed IPv6", "[::1]", 8443, false, "[::1]:8443", false}, + {"port 1", "h.databricks.com", 1, false, "h.databricks.com:1", false}, + {"port 65535", "h.databricks.com", 65535, false, "h.databricks.com:65535", false}, + {"embedded port equal", "h.databricks.com:8443", 8443, true, "h.databricks.com:8443", false}, + {"embedded port equal, IPv6", "[::1]:8443", 8443, true, "[::1]:8443", false}, + {"unset 443 defers to embedded", "h.databricks.com:8443", 443, false, "h.databricks.com:8443", false}, + {"explicit 443 matches embedded", "h.databricks.com:443", 443, true, "h.databricks.com:443", false}, + {"explicit 443 conflicts with embedded", "h.databricks.com:8443", 443, true, "", true}, + {"negative port", "h.databricks.com", -1, false, "", true}, + {"port above range", "h.databricks.com", 65536, false, "", true}, + {"embedded port differs", "h.databricks.com:7000", 8443, false, "", true}, + {"embedded port differs, IPv6", "[::1]:7000", 8443, false, "", true}, + {"embedded port not a number", "h.databricks.com:notaport", 8443, false, "", true}, + {"embedded port out of range", "h.databricks.com:70000", 0, false, "", true}, + {"embedded port zero", "h.databricks.com:0", 0, false, "", true}, + {"unclosed IPv6 bracket", "[::1", 8443, false, "", true}, + {"text after IPv6 bracket", "[::1]x", 8443, false, "", true}, + {"stray closing bracket", "h.databricks.com]", 8443, false, "", true}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + err := validateKernelHostPort(tc.host, tc.port, tc.explicit) + if tc.wantErr { + if !errors.Is(err, dbsqlerr.ErrInvalidKernelConfig) { + t.Errorf("err = %v, want ErrInvalidKernelConfig", err) + } + return + } + if err != nil { + t.Errorf("unexpected err %v", err) + } + if got := kernelHostWithPort(tc.host, tc.port); got != tc.wantHost { + t.Errorf("kernelHostWithPort(%q, %d) = %q, want %q", tc.host, tc.port, got, tc.wantHost) + } + }) + } + + t.Run("explicit WithPort(443) conflicting with an embedded port is rejected", func(t *testing.T) { + c := baseKernelConfig() + c.Host = "h.databricks.com:8443" + WithPort(443)(c) + if _, err := validateKernelConfig(c); !errors.Is(err, dbsqlerr.ErrInvalidKernelConfig) { + t.Errorf("err = %v, want ErrInvalidKernelConfig", err) + } + }) + + t.Run("kernel Host: unset port keeps the embedded port, custom port folds in", func(t *testing.T) { + c := baseKernelConfig() + c.Host = "h.databricks.com:8443" + kauth, err := validateKernelConfig(c) + if err != nil || buildKernelConfig(c, kauth).Host != "h.databricks.com:8443" { + t.Errorf("embedded port: err=%v host=%q", err, buildKernelConfig(c, kauth).Host) + } + c = baseKernelConfig() + WithPort(8443)(c) + kauth, err = validateKernelConfig(c) + if err != nil || buildKernelConfig(c, kauth).Host != "h.databricks.com:8443" { + t.Errorf("WithPort(8443): err=%v host=%q", err, buildKernelConfig(c, kauth).Host) + } + }) +} From ba84162a6778679fd3ee504df71f74b1fe62c7fb Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 18:26:48 +0000 Subject: [PATCH 2/7] fix(kernel): proxy lookup uses the kernel host authority proxyForEndpointFunc appended the port to cfg.Host even when the host already carried one, so host:8443 with Port 8443 built an invalid endpoint, the proxy was dropped and HTTPS_PROXY was ignored. It now uses the same authority as kernelHostWithPort, which leaves an embedded port alone and brackets IPv6 literals. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- kernel_proxy.go | 7 +++---- kernel_proxy_test.go | 31 +++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 4 deletions(-) diff --git a/kernel_proxy.go b/kernel_proxy.go index 9975e0ab..1c37b445 100644 --- a/kernel_proxy.go +++ b/kernel_proxy.go @@ -43,10 +43,9 @@ func proxyForEndpointFunc(cfg *config.Config, resolve func(*http.Request) (*url. if scheme == "" { scheme = "https" } - endpoint := fmt.Sprintf("%s://%s", scheme, cfg.Host) - if cfg.Port != 0 { - endpoint = fmt.Sprintf("%s://%s:%d", scheme, cfg.Host, cfg.Port) - } + // Same authority the kernel connects to: an embedded port is not appended twice + // and an IPv6 literal is bracketed (kernelHostWithPort). + endpoint := fmt.Sprintf("%s://%s", scheme, kernelHostWithPort(cfg.Host, cfg.Port)) req, err := http.NewRequest(http.MethodPost, endpoint, nil) if err != nil { return "" diff --git a/kernel_proxy_test.go b/kernel_proxy_test.go index 3bfed871..0e16f190 100644 --- a/kernel_proxy_test.go +++ b/kernel_proxy_test.go @@ -55,6 +55,37 @@ func TestProxyForEndpoint(t *testing.T) { resolve: func(*http.Request) (*url.URL, error) { return proxyURL, nil }, want: "http://corp-proxy:3128", }, + { + name: "embedded port equal to Port is not doubled", + cfg: func() *config.Config { + c := config.WithDefaults() + c.Host = "my-workspace.databricks.com:8443" + c.Port = 8443 + c.HTTPPath = "/sql/1.0/warehouses/abc" + return c + }(), + resolve: func(*http.Request) (*url.URL, error) { return proxyURL, nil }, + want: "http://corp-proxy:3128", + }, + { + // An unbracketed IPv6 literal with a port must be bracketed in the endpoint, + // the same way the kernel Host is built. + name: "IPv6 literal with Port is bracketed", + cfg: func() *config.Config { + c := config.WithDefaults() + c.Host = "::1" + c.Port = 8443 + c.HTTPPath = "/sql/1.0/warehouses/abc" + return c + }(), + resolve: func(req *http.Request) (*url.URL, error) { + if req.URL.Host != "[::1]:8443" { + t.Errorf("endpoint host = %q, want %q", req.URL.Host, "[::1]:8443") + } + return proxyURL, nil + }, + want: "http://corp-proxy:3128", + }, { name: "host excluded by NO_PROXY -> direct", cfg: validCfg(), From 263c7415951fdd8c18a67736e0694d25393dc2dc Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 20:10:05 +0000 Subject: [PATCH 3/7] test(kernel): cover custom port on Reyden recovery; clarify port range Address the review nits on the kernel custom-port change: - The out-of-range WithPort error now says 0 is accepted as the default. - PortExplicit is classified as "validated", not "forwarded": it is consumed by validateKernelHostPort and never sent to the kernel. - A Reyden recovery test with WithPort(8443) asserts that the kernel Host is host:8443 after the Thrift KP001 rejection. The CHANGELOG entry mentions the recovery path. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 +- connector_reyden_test.go | 23 +++++++++++++++++++++++ kernel_config.go | 2 +- kernel_config_test.go | 2 +- 4 files changed, 26 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 29847d14..fec9afee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased (v2.0.0 prototype) -- Honor `WithPort` / DSN port on the kernel backend: a non-default port is folded into the host (`host:port`, IPv6-safe) instead of being rejected, and a port that conflicts with one embedded in the host, or is out of range, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). +- Honor `WithPort` / DSN port on the kernel backend, including the automatic Reyden recovery onto the kernel: a non-default port is folded into the host (`host:port`, IPv6-safe) instead of being rejected, and a port that conflicts with one embedded in the host, or is out of range, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). - Fix the module-root package conflict when building, linting, or running Gazelle with the `tools` build tag (databricks/databricks-sql-go#162). - **Behavior change:** Report declared DECIMAL precision and scale through `sql.ColumnType.DecimalSize()` on Thrift and SEA, including zero-row results. Tools such as ORM migrators can now act on these qualifiers (databricks/databricks-sql-go#500). diff --git a/connector_reyden_test.go b/connector_reyden_test.go index 858d8ca3..0a89b3ea 100644 --- a/connector_reyden_test.go +++ b/connector_reyden_test.go @@ -216,6 +216,29 @@ func TestReydenReactiveRecovery(t *testing.T) { assert.True(t, warehouse_cache.IsKnownReyden(tt.host, tt.warehouseID), "warehouse should be marked Reyden after the rejection") }) + + t.Run("recovery onto the kernel keeps a custom WithPort", func(t *testing.T) { + defer warehouse_cache.ClearCache() + + tt := NewTestReydenFallback() + var kernelHost string + conn := tt.makeConnector( + func(ctx context.Context, cfg *config.Config, client *http.Client) (backend.Backend, error) { + return &fakeThriftBackend{openSessionErr: dbsqlerrint.NewReydenThriftUnsupportedError( + "Lakehouse/RT is not supported for Thrift protocol")}, nil + }, + func(ctx context.Context, cfg *config.Config) (backend.Backend, error) { + require.NoError(t, validateKernelHostPort(cfg.Host, cfg.Port, cfg.PortExplicit)) + kernelHost = buildKernelConfig(cfg, kernel.Auth{Mode: kernel.AuthPAT, Token: "test-token"}).Host + return &fakeKernelBackend{}, nil + }, + ) + WithPort(8443)(conn.cfg) + + _, _, err := conn.openSessionWithReydenFallback(context.Background()) + require.NoError(t, err) + assert.Equal(t, "reyden.example.com:8443", kernelHost) + }) } func TestReydenPreCheck(t *testing.T) { diff --git a/kernel_config.go b/kernel_config.go index c6c3a4dd..dabc3b9a 100644 --- a/kernel_config.go +++ b/kernel_config.go @@ -251,7 +251,7 @@ func unbalancedBrackets(host string) bool { // the DSN set the port. WithDefaults fills 443 otherwise, so an unset 443 defers to the embedded port. func validateKernelHostPort(host string, port int, explicit bool) error { if port < 0 || port > 65535 { - return fmt.Errorf("databricks: WithPort %d is out of range (want 1..65535): %w", port, dbsqlerr.ErrInvalidKernelConfig) + return fmt.Errorf("databricks: WithPort %d is out of range (want 1..65535, or 0 for the default): %w", port, dbsqlerr.ErrInvalidKernelConfig) } if unbalancedBrackets(host) { return fmt.Errorf("databricks: host %q has unbalanced IPv6 brackets: %w", host, dbsqlerr.ErrInvalidKernelConfig) diff --git a/kernel_config_test.go b/kernel_config_test.go index 30282689..5e49d5e5 100644 --- a/kernel_config_test.go +++ b/kernel_config_test.go @@ -445,7 +445,7 @@ var kernelConfigFieldDisposition = map[string]string{ "QueryTimeout": "rejected", // when > 0 (WithTimeout) "Protocol": "rejected", // kernel is https-only; non-default rejected "Port": "forwarded", // folded into the kernel Host by kernelHostWithPort; validated by validateKernelHostPort - "PortExplicit": "forwarded", // set by WithPort / the DSN; consumed by validation, not sent to the kernel + "PortExplicit": "validated", // set by WithPort / the DSN; consumed by validation, not sent to the kernel "Transport": "rejected", // custom RoundTripper; kernel uses its own HTTP stack, so reject rather than drop // Forwarded to the kernel's HTTP retry config via kernelRetryConfig → From 7bc6a2ae823670091c5b39201520d647a37b2a9c Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 23:13:04 +0000 Subject: [PATCH 4/7] kernel: normalise trailing slash and embedded-port precedence for WithPort Fold a trailing "/" in the host before any port handling, matching the kernel's normalise_host, so WithServerHostname("https://h/") plus WithPort(8443) connects to h:8443 instead of h/:8443, and "https://h:8443/" keeps working. Port precedence now matches the Node and Python kernel paths: - A port embedded in the host wins over a differing WithPort. The driver logs a warning naming both ports and connects to the embedded port. - A zero (unset) WithPort means the default, with no :443 added. - A host with a path, query or fragment after the trimmed "/" is rejected with ErrInvalidKernelConfig. Base also rejected it: the kernel builds {host}/api/2.0/sql, so h/x sent requests to /x/api/2.0/sql. - Out-of-range WithPort and out-of-range or malformed embedded ports stay rejected. Thrift also fails on them: an embedded port plus the default 443 yields an unparseable https://h:8443:443 URL, -1 is an invalid port, and 70000 fails to dial. Base behavior (origin/main): validateKernelConfigContext rejected any WithPort other than 0 or 443 with ErrNotSupportedByKernel, and passed the host through unchanged, so an embedded port connected to that port. Tests: table cases for h/, h:8443/, h/x, [::1]/, embedded-vs-WithPort (warn, embedded wins), explicit 443 against an embedded port, and an invalid port (still rejected). Proxy endpoint test for a trailing-slash host. Kernel loopback probe: embedded port dialed, explicit port never. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 +- CONNECTION_PARAMETERS.md | 2 +- README.md | 2 +- kernel_config.go | 50 ++++++++++++++++++++++++++++++++-------- kernel_config_test.go | 50 +++++++++++++++++++++++++++++++++++----- kernel_proxy_test.go | 19 +++++++++++++++ 6 files changed, 107 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fec9afee..9bf41163 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased (v2.0.0 prototype) -- Honor `WithPort` / DSN port on the kernel backend, including the automatic Reyden recovery onto the kernel: a non-default port is folded into the host (`host:port`, IPv6-safe) instead of being rejected, and a port that conflicts with one embedded in the host, or is out of range, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). +- Honor `WithPort` / DSN port on the kernel backend, including the automatic Reyden recovery onto the kernel: a non-default port is folded into the host (`host:port`, IPv6-safe) instead of being rejected. A port embedded in the host wins over a differing `WithPort` (a warning is logged), a zero port means the default, and a trailing `/` on the host is ignored. An out-of-range port, or a host with a path, query or fragment, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). - Fix the module-root package conflict when building, linting, or running Gazelle with the `tools` build tag (databricks/databricks-sql-go#162). - **Behavior change:** Report declared DECIMAL precision and scale through `sql.ColumnType.DecimalSize()` on Thrift and SEA, including zero-row results. Tools such as ORM migrators can now act on these qualifiers (databricks/databricks-sql-go#500). diff --git a/CONNECTION_PARAMETERS.md b/CONNECTION_PARAMETERS.md index 7d06a065..b7013f28 100644 --- a/CONNECTION_PARAMETERS.md +++ b/CONNECTION_PARAMETERS.md @@ -40,7 +40,7 @@ wire. They are listed under [Kernel client-result confs](#kernel-client-result-c |---|---|:---:|:---:|---|---| | *(host)* | `WithServerHostname` | ✅ | ✅ | *(required)* | Workspace hostname. | | *(path)* | `WithHTTPPath` | ✅ | ✅ | required on Thrift except localhost; required on kernel unless `warehouseId` is set | Warehouse/endpoint HTTP path. On the kernel path, a bare warehouse id can be used instead. | -| *(port)* | `WithPort` | ✅ | ✅ | `443` for connector options; required in a DSN | Kernel folds a non-default port into the host it connects to; an out-of-range or conflicting port (including an explicit `WithPort(443)` against a host port) is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | +| *(port)* | `WithPort` | ✅ | ✅ | `443` for connector options; required in a DSN | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged and the embedded port is used); an out-of-range port, or a host with a path, query or fragment, is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | | *(scheme)* | *(via `WithServerHostname`)* | ✅ | ⚠️ | `https` | Kernel accepts `https` only and rejects `http`. A bare `localhost` passed to `WithServerHostname` selects `http`; a scheme-less DSN, including localhost, is prefixed with `https`. | | `warehouseId` | `WithWarehouseID` | ⚠️ | ✅ | | Bare warehouse id. The kernel normally prefers it over the HTTP path; a canonical path carrying `?o=` takes precedence so workspace-org routing is preserved. The Thrift backend **silently ignores** it. | | `catalog` | `WithInitialNamespace` | ✅ | ✅ | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. | diff --git a/README.md b/README.md index 8f784bfd..df43fa70 100644 --- a/README.md +++ b/README.md @@ -240,7 +240,7 @@ session parameter on both backends. |---|---|---|---|---| | *(host)* | `WithServerHostname` | Both | *(required)* | Workspace hostname. | | *(path)* | `WithHTTPPath` | Both | *(required)* | Warehouse/endpoint HTTP path. | -| *(port)* | `WithPort` | Both | `443` | Kernel folds a non-default port into the host it connects to; an out-of-range or conflicting port is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | +| *(port)* | `WithPort` | Both | `443` | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged); an out-of-range port, or a host with a path, query or fragment, is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | | `warehouseId` | `WithWarehouseID` | SEA only | | Bare warehouse id; the kernel routes by it (preferred over the HTTP path). **The Thrift backend ignores it.** | | `catalog` | `WithInitialNamespace` | Both | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. | | `schema` | `WithInitialNamespace` | Both | | Initial schema. Kernel applies it post-connect via `USE SCHEMA`. | diff --git a/kernel_config.go b/kernel_config.go index dabc3b9a..b63f6673 100644 --- a/kernel_config.go +++ b/kernel_config.go @@ -16,6 +16,7 @@ import ( "github.com/databricks/databricks-sql-go/internal/backend/kernel" "github.com/databricks/databricks-sql-go/internal/client" "github.com/databricks/databricks-sql-go/internal/config" + "github.com/databricks/databricks-sql-go/logger" "github.com/databricks/databricks-sql-go/telemetry" ) @@ -79,10 +80,12 @@ func validateKernelConfigContext(ctx context.Context, cfg *config.Config) (kerne return kernel.Auth{}, fmt.Errorf("databricks: a non-https protocol is %w "+ "(it connects over https); use the default (Thrift) backend", dbsqlerr.ErrNotSupportedByKernel) } - // Port: the kernel folds host:port (see kernelHostWithPort); reject a bad port or a conflict here. + // Port: the kernel folds host:port (see kernelHostWithPort); reject a bad port or host here. An + // embedded host port wins over a differing WithPort, with a warning rather than an error. if err := validateKernelHostPort(cfg.Host, cfg.Port, cfg.PortExplicit); err != nil { return kernel.Auth{}, err } + warnEmbeddedPortWins(cfg.Host, cfg.Port, cfg.PortExplicit) // Transport (WithTransport, a custom http.RoundTripper carrying a custom CA // bundle / mTLS / proxy): the kernel uses its own Rust HTTP stack below the C // ABI and never sees a Go RoundTripper, so a custom Transport would be silently @@ -246,13 +249,26 @@ func unbalancedBrackets(host string) bool { return end < 0 || (end+1 < len(host) && host[end+1] != ':') } -// validateKernelHostPort rejects an out-of-range port, a host with unbalanced IPv6 brackets, and a -// supplied port that disagrees with a port embedded in the host. explicit is true when WithPort or -// the DSN set the port. WithDefaults fills 443 otherwise, so an unset 443 defers to the embedded port. +// normaliseKernelHost trims trailing "/" the way the kernel's normalise_host does, so the port +// logic below sees the same authority the kernel connects to. +func normaliseKernelHost(host string) string { + return strings.TrimRight(host, "/") +} + +// validateKernelHostPort rejects an out-of-range WithPort, a host with a path, query or fragment, +// unbalanced IPv6 brackets, and an out-of-range or malformed port embedded in the host. explicit is +// true when WithPort or the DSN set the port. WithDefaults fills 443 otherwise. A port embedded in +// the host wins over WithPort (see warnEmbeddedPortWins), so a differing WithPort is not an error. func validateKernelHostPort(host string, port int, explicit bool) error { if port < 0 || port > 65535 { return fmt.Errorf("databricks: WithPort %d is out of range (want 1..65535, or 0 for the default): %w", port, dbsqlerr.ErrInvalidKernelConfig) } + host = normaliseKernelHost(host) + // A path, query or fragment after the host made the kernel URL point at the wrong place + // (the kernel appends /api/2.0/sql to the host), so it was never accepted. + if strings.ContainsAny(host, "/?#") { + return fmt.Errorf("databricks: host %q has a path, query or fragment: %w", host, dbsqlerr.ErrInvalidKernelConfig) + } if unbalancedBrackets(host) { return fmt.Errorf("databricks: host %q has unbalanced IPv6 brackets: %w", host, dbsqlerr.ErrInvalidKernelConfig) } @@ -264,16 +280,32 @@ func validateKernelHostPort(host string, port int, explicit bool) error { if err != nil || embedded < 1 || embedded > 65535 { return fmt.Errorf("databricks: host %q carries an invalid port %q (want 1..65535): %w", host, p, dbsqlerr.ErrInvalidKernelConfig) } - if supplied := port != 0 && (explicit || port != 443); supplied && port != embedded { - return fmt.Errorf("databricks: host %q carries port %d but WithPort is %d: %w", host, embedded, port, dbsqlerr.ErrInvalidKernelConfig) - } return nil } +// warnEmbeddedPortWins logs when a supplied port (WithPort or the DSN) differs from the port embedded +// in the host. The embedded port is used, as on the kernel path before WithPort was honored. +func warnEmbeddedPortWins(host string, port int, explicit bool) { + if port == 0 || (!explicit && port == 443) { + return + } + _, p, err := net.SplitHostPort(normaliseKernelHost(host)) + if err != nil { + return + } + embedded, err := strconv.Atoi(p) + if err != nil || embedded == port { + return + } + logger.Warn().Msgf("databricks: the host carries port %d, which differs from WithPort %d; connecting to port %d from the host", + embedded, port, embedded) +} + // kernelHostWithPort returns the Host the kernel connects to. The kernel has no port setter but -// keeps a `host:port` authority, so a non-default port is folded in (IPv6-safe). 0 and 443 add -// nothing, and a host that already carries a port is returned unchanged. +// keeps a `host:port` authority, so a non-default port is folded in (IPv6-safe). A trailing "/" is +// trimmed first. 0 and 443 add nothing, and a host that already carries a port is returned unchanged. func kernelHostWithPort(host string, port int) string { + host = normaliseKernelHost(host) if port == 0 || port == 443 { return host } diff --git a/kernel_config_test.go b/kernel_config_test.go index 5e49d5e5..885a1099 100644 --- a/kernel_config_test.go +++ b/kernel_config_test.go @@ -998,11 +998,20 @@ func TestKernelCustomPort(t *testing.T) { {"embedded port equal, IPv6", "[::1]:8443", 8443, true, "[::1]:8443", false}, {"unset 443 defers to embedded", "h.databricks.com:8443", 443, false, "h.databricks.com:8443", false}, {"explicit 443 matches embedded", "h.databricks.com:443", 443, true, "h.databricks.com:443", false}, - {"explicit 443 conflicts with embedded", "h.databricks.com:8443", 443, true, "", true}, + {"explicit 443 differs from embedded: embedded wins", "h.databricks.com:8443", 443, true, "h.databricks.com:8443", false}, + {"embedded port differs: embedded wins", "h.databricks.com:7000", 8443, false, "h.databricks.com:7000", false}, + {"embedded port differs, IPv6: embedded wins", "[::1]:7000", 8443, false, "[::1]:7000", false}, + {"trailing slash on bare host", "h.databricks.com/", 8443, false, "h.databricks.com:8443", false}, + {"trailing slash on bare host, default port", "h.databricks.com/", 443, false, "h.databricks.com", false}, + {"trailing slash with embedded port, default port", "h.databricks.com:8443/", 443, false, "h.databricks.com:8443", false}, + {"trailing slash with embedded port, equal port", "h.databricks.com:8443/", 8443, true, "h.databricks.com:8443", false}, + {"trailing slash, IPv6", "[::1]/", 8443, false, "[::1]:8443", false}, + {"path after host", "h.databricks.com/x", 8443, false, "", true}, + {"path after host, default port", "h.databricks.com/x", 443, false, "", true}, + {"query after host", "h.databricks.com?x", 8443, false, "", true}, + {"fragment after host", "h.databricks.com#x", 8443, false, "", true}, {"negative port", "h.databricks.com", -1, false, "", true}, {"port above range", "h.databricks.com", 65536, false, "", true}, - {"embedded port differs", "h.databricks.com:7000", 8443, false, "", true}, - {"embedded port differs, IPv6", "[::1]:7000", 8443, false, "", true}, {"embedded port not a number", "h.databricks.com:notaport", 8443, false, "", true}, {"embedded port out of range", "h.databricks.com:70000", 0, false, "", true}, {"embedded port zero", "h.databricks.com:0", 0, false, "", true}, @@ -1028,12 +1037,41 @@ func TestKernelCustomPort(t *testing.T) { }) } - t.Run("explicit WithPort(443) conflicting with an embedded port is rejected", func(t *testing.T) { + t.Run("explicit WithPort(443) differing from an embedded port connects to the embedded port", func(t *testing.T) { c := baseKernelConfig() c.Host = "h.databricks.com:8443" WithPort(443)(c) - if _, err := validateKernelConfig(c); !errors.Is(err, dbsqlerr.ErrInvalidKernelConfig) { - t.Errorf("err = %v, want ErrInvalidKernelConfig", err) + kauth, err := validateKernelConfig(c) + if err != nil { + t.Fatalf("unexpected err %v", err) + } + if got := buildKernelConfig(c, kauth).Host; got != "h.databricks.com:8443" { + t.Errorf("kernel Host = %q, want the embedded port h.databricks.com:8443", got) + } + }) + + t.Run("WithServerHostname with a trailing slash plus WithPort folds to host:port", func(t *testing.T) { + c := baseKernelConfig() + WithServerHostname("https://h.databricks.com/")(c) + WithPort(8443)(c) + kauth, err := validateKernelConfig(c) + if err != nil { + t.Fatalf("unexpected err %v", err) + } + if got := buildKernelConfig(c, kauth).Host; got != "h.databricks.com:8443" { + t.Errorf("kernel Host = %q, want h.databricks.com:8443", got) + } + }) + + t.Run("WithServerHostname with an embedded port and a trailing slash is still accepted", func(t *testing.T) { + c := baseKernelConfig() + WithServerHostname("https://h.databricks.com:8443/")(c) + kauth, err := validateKernelConfig(c) + if err != nil { + t.Fatalf("unexpected err %v", err) + } + if got := buildKernelConfig(c, kauth).Host; got != "h.databricks.com:8443" { + t.Errorf("kernel Host = %q, want h.databricks.com:8443", got) } }) diff --git a/kernel_proxy_test.go b/kernel_proxy_test.go index 0e16f190..9316e470 100644 --- a/kernel_proxy_test.go +++ b/kernel_proxy_test.go @@ -111,6 +111,25 @@ func TestProxyForEndpoint(t *testing.T) { }, want: "", }, + { + // A trailing "/" (WithServerHostname("https://h/")) is trimmed before the port is + // folded, so the endpoint is h:8443, not h/:8443. + name: "trailing slash host with Port folds to host:port", + cfg: func() *config.Config { + c := config.WithDefaults() + c.Host = "my-workspace.databricks.com/" + c.Port = 8443 + c.HTTPPath = "/sql/1.0/warehouses/abc" + return c + }(), + resolve: func(req *http.Request) (*url.URL, error) { + if req.URL.Host != "my-workspace.databricks.com:8443" { + t.Errorf("endpoint host = %q, want %q", req.URL.Host, "my-workspace.databricks.com:8443") + } + return proxyURL, nil + }, + want: "http://corp-proxy:3128", + }, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { From 5615750373d4e90ad008c396bb73df8dd73f255c Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 23:21:23 +0000 Subject: [PATCH 5/7] kernel: reject paths before the embedded-port parse; no warning for explicit 443 - A host with a path or query is rejected before the embedded-port parse, so "h.io/path:8443" with an explicit port cannot slip through. The old code passed the host verbatim to the kernel, which builds {host}/api/2.0/sql, so these configs never connected. - The differing-port warning is skipped for an explicit 443 (the default). - Thrift is unchanged. TestThriftEndpointUnchanged pins its current endpoint strings for the same port inputs, including the embedded-port-plus-443 case. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- internal/config/thrift_endpoint_test.go | 32 +++++++++++++++++++++++++ kernel_config.go | 11 +++++---- kernel_config_test.go | 4 ++++ 3 files changed, 42 insertions(+), 5 deletions(-) create mode 100644 internal/config/thrift_endpoint_test.go diff --git a/internal/config/thrift_endpoint_test.go b/internal/config/thrift_endpoint_test.go new file mode 100644 index 00000000..efdcf13b --- /dev/null +++ b/internal/config/thrift_endpoint_test.go @@ -0,0 +1,32 @@ +package config + +import "testing" + +// TestThriftEndpointUnchanged pins the Thrift endpoint for the port inputs the kernel custom-port +// change decides on. Thrift is not changed by that work, so these strings are what it builds today. +// An embedded host port plus the default 443 yields an unparseable URL on Thrift; that is the baseline. +func TestThriftEndpointUnchanged(t *testing.T) { + cases := []struct { + name string + host string + port int + want string + }{ + {"bare host, default port", "h.io", 443, "https://h.io:443/sql/1.0/warehouses/x"}, + {"bare host, WithPort", "h.io", 8443, "https://h.io:8443/sql/1.0/warehouses/x"}, + {"embedded port, default port", "h.io:8443", 443, "https://h.io:8443:443/sql/1.0/warehouses/x"}, + {"embedded port, WithPort", "h.io:8443", 8443, "https://h.io:8443:8443/sql/1.0/warehouses/x"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + c := &Config{Protocol: "https", Host: tc.host, Port: tc.port, HTTPPath: "/sql/1.0/warehouses/x"} + got, err := c.ToEndpointURL() + if err != nil { + t.Fatalf("unexpected err %v", err) + } + if got != tc.want { + t.Errorf("ToEndpointURL() = %q, want %q", got, tc.want) + } + }) + } +} diff --git a/kernel_config.go b/kernel_config.go index b63f6673..0c1dd280 100644 --- a/kernel_config.go +++ b/kernel_config.go @@ -85,7 +85,7 @@ func validateKernelConfigContext(ctx context.Context, cfg *config.Config) (kerne if err := validateKernelHostPort(cfg.Host, cfg.Port, cfg.PortExplicit); err != nil { return kernel.Auth{}, err } - warnEmbeddedPortWins(cfg.Host, cfg.Port, cfg.PortExplicit) + warnEmbeddedPortWins(cfg.Host, cfg.Port) // Transport (WithTransport, a custom http.RoundTripper carrying a custom CA // bundle / mTLS / proxy): the kernel uses its own Rust HTTP stack below the C // ABI and never sees a Go RoundTripper, so a custom Transport would be silently @@ -283,10 +283,11 @@ func validateKernelHostPort(host string, port int, explicit bool) error { return nil } -// warnEmbeddedPortWins logs when a supplied port (WithPort or the DSN) differs from the port embedded -// in the host. The embedded port is used, as on the kernel path before WithPort was honored. -func warnEmbeddedPortWins(host string, port int, explicit bool) { - if port == 0 || (!explicit && port == 443) { +// warnEmbeddedPortWins logs when a non-default port (WithPort or the DSN) differs from the port embedded +// in the host. The embedded port is used, as on the kernel path before WithPort was honored. An +// explicit 443 is the default and is not warned about. +func warnEmbeddedPortWins(host string, port int) { + if port == 0 || port == 443 { return } _, p, err := net.SplitHostPort(normaliseKernelHost(host)) diff --git a/kernel_config_test.go b/kernel_config_test.go index 885a1099..d60865b7 100644 --- a/kernel_config_test.go +++ b/kernel_config_test.go @@ -1007,6 +1007,10 @@ func TestKernelCustomPort(t *testing.T) { {"trailing slash with embedded port, equal port", "h.databricks.com:8443/", 8443, true, "h.databricks.com:8443", false}, {"trailing slash, IPv6", "[::1]/", 8443, false, "[::1]:8443", false}, {"path after host", "h.databricks.com/x", 8443, false, "", true}, + {"path with embedded port and WithPort", "h.io/path:8443", 8443, true, "", true}, + {"path with embedded port, default port", "h.io/path:8443", 443, false, "", true}, + {"query before embedded port", "h.io?q=1:8443", 8443, true, "", true}, + {"explicit 443 with embedded port: embedded wins", "h.io:8443", 443, true, "h.io:8443", false}, {"path after host, default port", "h.databricks.com/x", 443, false, "", true}, {"query after host", "h.databricks.com?x", 8443, false, "", true}, {"fragment after host", "h.databricks.com#x", 8443, false, "", true}, From 52bd5ac9567549480c3bdee0e6179d67b360d7e5 Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 23:31:59 +0000 Subject: [PATCH 6/7] kernel: keep path hosts unchanged on the default port; drop PortExplicit - A host with a path, query or fragment is now rejected only when a non-default WithPort would have to be folded into it. The check still runs after the trailing "/" trim and before the embedded-port parse. With no port or 443 the host goes to the kernel unchanged, as on the base branch, where the kernel keeps a path prefix ({host}/api/2.0/sql). The base branch rejected every non-default WithPort, so nothing that worked before is rejected now. - Remove the internal PortExplicit field. Once an explicit 443 stopped warning, nothing read it, so WithPort, ParseDSN, DeepCopy and their tests are back to the base branch. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 +- CONNECTION_PARAMETERS.md | 2 +- README.md | 2 +- connector.go | 1 - connector_reyden_test.go | 2 +- connector_test.go | 1 - driver_test.go | 1 - internal/config/config.go | 11 ++--- internal/config/config_test.go | 25 ---------- kernel_config.go | 23 ++++++---- kernel_config_test.go | 83 ++++++++++++++++------------------ 11 files changed, 61 insertions(+), 92 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9bf41163..bc66df21 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased (v2.0.0 prototype) -- Honor `WithPort` / DSN port on the kernel backend, including the automatic Reyden recovery onto the kernel: a non-default port is folded into the host (`host:port`, IPv6-safe) instead of being rejected. A port embedded in the host wins over a differing `WithPort` (a warning is logged), a zero port means the default, and a trailing `/` on the host is ignored. An out-of-range port, or a host with a path, query or fragment, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). +- Honor `WithPort` / DSN port on the kernel backend, including the automatic Reyden recovery onto the kernel: a non-default port is folded into the host (`host:port`, IPv6-safe) instead of being rejected. A port embedded in the host wins over a differing `WithPort` (a warning is logged), a zero port means the default, and a trailing `/` on the host is ignored. An out-of-range port, or a non-default port with a host that has a path, query or fragment, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). - Fix the module-root package conflict when building, linting, or running Gazelle with the `tools` build tag (databricks/databricks-sql-go#162). - **Behavior change:** Report declared DECIMAL precision and scale through `sql.ColumnType.DecimalSize()` on Thrift and SEA, including zero-row results. Tools such as ORM migrators can now act on these qualifiers (databricks/databricks-sql-go#500). diff --git a/CONNECTION_PARAMETERS.md b/CONNECTION_PARAMETERS.md index b7013f28..3d09179b 100644 --- a/CONNECTION_PARAMETERS.md +++ b/CONNECTION_PARAMETERS.md @@ -40,7 +40,7 @@ wire. They are listed under [Kernel client-result confs](#kernel-client-result-c |---|---|:---:|:---:|---|---| | *(host)* | `WithServerHostname` | ✅ | ✅ | *(required)* | Workspace hostname. | | *(path)* | `WithHTTPPath` | ✅ | ✅ | required on Thrift except localhost; required on kernel unless `warehouseId` is set | Warehouse/endpoint HTTP path. On the kernel path, a bare warehouse id can be used instead. | -| *(port)* | `WithPort` | ✅ | ✅ | `443` for connector options; required in a DSN | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged and the embedded port is used); an out-of-range port, or a host with a path, query or fragment, is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | +| *(port)* | `WithPort` | ✅ | ✅ | `443` for connector options; required in a DSN | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged and the embedded port is used); an out-of-range port, or a non-default port with a host that has a path, query or fragment, is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | | *(scheme)* | *(via `WithServerHostname`)* | ✅ | ⚠️ | `https` | Kernel accepts `https` only and rejects `http`. A bare `localhost` passed to `WithServerHostname` selects `http`; a scheme-less DSN, including localhost, is prefixed with `https`. | | `warehouseId` | `WithWarehouseID` | ⚠️ | ✅ | | Bare warehouse id. The kernel normally prefers it over the HTTP path; a canonical path carrying `?o=` takes precedence so workspace-org routing is preserved. The Thrift backend **silently ignores** it. | | `catalog` | `WithInitialNamespace` | ✅ | ✅ | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. | diff --git a/README.md b/README.md index df43fa70..0dae21b1 100644 --- a/README.md +++ b/README.md @@ -240,7 +240,7 @@ session parameter on both backends. |---|---|---|---|---| | *(host)* | `WithServerHostname` | Both | *(required)* | Workspace hostname. | | *(path)* | `WithHTTPPath` | Both | *(required)* | Warehouse/endpoint HTTP path. | -| *(port)* | `WithPort` | Both | `443` | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged); an out-of-range port, or a host with a path, query or fragment, is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | +| *(port)* | `WithPort` | Both | `443` | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged); an out-of-range port, or a non-default port with a host that has a path, query or fragment, is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | | `warehouseId` | `WithWarehouseID` | SEA only | | Bare warehouse id; the kernel routes by it (preferred over the HTTP path). **The Thrift backend ignores it.** | | `catalog` | `WithInitialNamespace` | Both | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. | | `schema` | `WithInitialNamespace` | Both | | Initial schema. Kernel applies it post-connect via `USE SCHEMA`. | diff --git a/connector.go b/connector.go index 34c1350d..1b72c619 100644 --- a/connector.go +++ b/connector.go @@ -367,7 +367,6 @@ func parseHostName(host string) (protocol, hostname string) { func WithPort(port int) ConnOption { return func(c *config.Config) { c.Port = port - c.PortExplicit = true } } diff --git a/connector_reyden_test.go b/connector_reyden_test.go index 0a89b3ea..d789631c 100644 --- a/connector_reyden_test.go +++ b/connector_reyden_test.go @@ -228,7 +228,7 @@ func TestReydenReactiveRecovery(t *testing.T) { "Lakehouse/RT is not supported for Thrift protocol")}, nil }, func(ctx context.Context, cfg *config.Config) (backend.Backend, error) { - require.NoError(t, validateKernelHostPort(cfg.Host, cfg.Port, cfg.PortExplicit)) + require.NoError(t, validateKernelHostPort(cfg.Host, cfg.Port)) kernelHost = buildKernelConfig(cfg, kernel.Auth{Mode: kernel.AuthPAT, Token: "test-token"}).Host return &fakeKernelBackend{}, nil }, diff --git a/connector_test.go b/connector_test.go index 2ecb0266..684a8646 100644 --- a/connector_test.go +++ b/connector_test.go @@ -174,7 +174,6 @@ func TestNewConnector(t *testing.T) { expectedUserConfig := config.UserConfig{ Host: host, Port: port, - PortExplicit: true, Protocol: "https", AccessToken: accessToken, Authenticator: &pat.PATAuth{AccessToken: accessToken}, diff --git a/driver_test.go b/driver_test.go index 5aa6f4e8..53aba0eb 100644 --- a/driver_test.go +++ b/driver_test.go @@ -21,7 +21,6 @@ func TestOOpenConnector(t *testing.T) { expectedUserConfig := config.UserConfig{ Host: host, Port: port, - PortExplicit: true, Protocol: "https", AccessToken: accessToken, HTTPPath: "/" + httpPath, diff --git a/internal/config/config.go b/internal/config/config.go index 3d808970..f9c5143e 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -265,12 +265,9 @@ func (c *Config) DeepCopy() *Config { // UserConfig is the set of configurations exposed to users type UserConfig struct { - Protocol string - Host string // from databricks UI - Port int // from databricks UI - // PortExplicit is true once WithPort or the DSN supplied Port (WithDefaults fills 443 otherwise). - // Internal only: kernel validation uses it to reject an explicit port that conflicts with a host port. - PortExplicit bool + Protocol string + Host string // from databricks UI + Port int // from databricks UI HTTPPath string // from databricks UI Catalog string Schema string @@ -345,7 +342,6 @@ func (ucfg UserConfig) DeepCopy() UserConfig { Protocol: ucfg.Protocol, Host: ucfg.Host, Port: ucfg.Port, - PortExplicit: ucfg.PortExplicit, HTTPPath: ucfg.HTTPPath, Catalog: ucfg.Catalog, Schema: ucfg.Schema, @@ -459,7 +455,6 @@ func ParseDSN(dsn string) (UserConfig, error) { return UserConfig{}, dbsqlerrint.NewRequestError(context.TODO(), dbsqlerr.ErrInvalidDSNPort, err) } ucfg.Port = port - ucfg.PortExplicit = true ucfg.HTTPPath = parsedURL.Path diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 07c1bf7f..518ccaf2 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -38,7 +38,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, - PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", @@ -59,7 +58,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, - PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -80,7 +78,6 @@ func TestParseConfig(t *testing.T) { Protocol: "http", Host: "localhost", Port: 8080, - PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &noop.NoopAuth{}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -100,7 +97,6 @@ func TestParseConfig(t *testing.T) { Protocol: "http", Host: "localhost", Port: 8080, - PortExplicit: true, Authenticator: &noop.NoopAuth{}, MaxRows: defaultMaxRows, SessionParams: make(map[string]string), @@ -119,7 +115,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -141,7 +136,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -165,7 +159,6 @@ func TestParseConfig(t *testing.T) { Host: "example.cloud.databricks.com", Authenticator: &noop.NoopAuth{}, Port: 8000, - PortExplicit: true, MaxRows: defaultMaxRows, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", SessionParams: make(map[string]string), @@ -184,7 +177,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -206,7 +198,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -228,7 +219,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, @@ -250,7 +240,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -276,7 +265,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -298,7 +286,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -325,7 +312,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -356,7 +342,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, - PortExplicit: true, MaxRows: defaultMaxRows, AccessToken: "supersecret", Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, @@ -377,7 +362,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, - PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, QueryTimeout: 100 * time.Second, @@ -435,7 +419,6 @@ func TestParseConfig(t *testing.T) { args: args{dsn: "token:supersecret2@:443?catalog=default&schema=system&timeout=100&maxRows=1000"}, wantCfg: UserConfig{ Port: 443, - PortExplicit: true, Protocol: "https", AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, @@ -460,7 +443,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -485,7 +467,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -510,7 +491,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, AccessToken: "supersecret2", Authenticator: &pat.PATAuth{AccessToken: "supersecret2"}, HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", @@ -565,7 +545,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 8000, - PortExplicit: true, Authenticator: m2m.NewAuthenticator("client_id", "client_secret", "example.cloud.databricks.com"), HTTPPath: "/sql/1.0/endpoints/12346a5b5b0e123a", QueryTimeout: 100 * time.Second, @@ -592,7 +571,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, - PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", @@ -619,7 +597,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, - PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", @@ -648,7 +625,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, - PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", @@ -670,7 +646,6 @@ func TestParseConfig(t *testing.T) { Protocol: "https", Host: "example.cloud.databricks.com", Port: 443, - PortExplicit: true, MaxRows: defaultMaxRows, Authenticator: &pat.PATAuth{AccessToken: "supersecret"}, AccessToken: "supersecret", diff --git a/kernel_config.go b/kernel_config.go index 0c1dd280..d841d30b 100644 --- a/kernel_config.go +++ b/kernel_config.go @@ -82,7 +82,7 @@ func validateKernelConfigContext(ctx context.Context, cfg *config.Config) (kerne } // Port: the kernel folds host:port (see kernelHostWithPort); reject a bad port or host here. An // embedded host port wins over a differing WithPort, with a warning rather than an error. - if err := validateKernelHostPort(cfg.Host, cfg.Port, cfg.PortExplicit); err != nil { + if err := validateKernelHostPort(cfg.Host, cfg.Port); err != nil { return kernel.Auth{}, err } warnEmbeddedPortWins(cfg.Host, cfg.Port) @@ -255,19 +255,24 @@ func normaliseKernelHost(host string) string { return strings.TrimRight(host, "/") } -// validateKernelHostPort rejects an out-of-range WithPort, a host with a path, query or fragment, -// unbalanced IPv6 brackets, and an out-of-range or malformed port embedded in the host. explicit is -// true when WithPort or the DSN set the port. WithDefaults fills 443 otherwise. A port embedded in -// the host wins over WithPort (see warnEmbeddedPortWins), so a differing WithPort is not an error. -func validateKernelHostPort(host string, port int, explicit bool) error { +// validateKernelHostPort rejects an out-of-range WithPort, a non-default WithPort with a host that has +// a path, query or fragment, unbalanced IPv6 brackets, and an out-of-range or malformed port embedded +// in the host. WithDefaults fills 443 when no port is set. A port embedded in the host wins over +// WithPort (see warnEmbeddedPortWins), so a differing WithPort is not an error. +func validateKernelHostPort(host string, port int) error { if port < 0 || port > 65535 { return fmt.Errorf("databricks: WithPort %d is out of range (want 1..65535, or 0 for the default): %w", port, dbsqlerr.ErrInvalidKernelConfig) } host = normaliseKernelHost(host) - // A path, query or fragment after the host made the kernel URL point at the wrong place - // (the kernel appends /api/2.0/sql to the host), so it was never accepted. + // A path, query or fragment is checked before the embedded-port parse. With the default port the + // host goes to the kernel unchanged, as before (the kernel appends its API paths after it). A + // non-default port cannot be folded into such a host, and was rejected before, so reject it. if strings.ContainsAny(host, "/?#") { - return fmt.Errorf("databricks: host %q has a path, query or fragment: %w", host, dbsqlerr.ErrInvalidKernelConfig) + if port != 0 && port != 443 { + return fmt.Errorf("databricks: host %q has a path, query or fragment, so WithPort %d cannot be applied: %w", + host, port, dbsqlerr.ErrInvalidKernelConfig) + } + return nil } if unbalancedBrackets(host) { return fmt.Errorf("databricks: host %q has unbalanced IPv6 brackets: %w", host, dbsqlerr.ErrInvalidKernelConfig) diff --git a/kernel_config_test.go b/kernel_config_test.go index d60865b7..0926fb4b 100644 --- a/kernel_config_test.go +++ b/kernel_config_test.go @@ -445,7 +445,6 @@ var kernelConfigFieldDisposition = map[string]string{ "QueryTimeout": "rejected", // when > 0 (WithTimeout) "Protocol": "rejected", // kernel is https-only; non-default rejected "Port": "forwarded", // folded into the kernel Host by kernelHostWithPort; validated by validateKernelHostPort - "PortExplicit": "validated", // set by WithPort / the DSN; consumed by validation, not sent to the kernel "Transport": "rejected", // custom RoundTripper; kernel uses its own HTTP stack, so reject rather than drop // Forwarded to the kernel's HTTP retry config via kernelRetryConfig → @@ -973,59 +972,57 @@ func TestKernelRetryConfig(t *testing.T) { }) } -// TestKernelCustomPort covers WithPort on the kernel backend (AC-P1, AC-P3): a non-default port is -// folded into the kernel Host, 0 and 443 add nothing, and an out-of-range port, unbalanced IPv6 -// brackets, or a supplied port that disagrees with an embedded port is rejected with -// ErrInvalidKernelConfig. +// TestKernelCustomPort covers WithPort on the kernel backend: a non-default port is folded into the +// kernel Host, 0 and 443 add nothing, a port embedded in the host wins, a trailing "/" is trimmed, +// and a host with a path passes through unchanged unless a non-default port would be folded into it. +// An out-of-range port or unbalanced IPv6 brackets are rejected with ErrInvalidKernelConfig. func TestKernelCustomPort(t *testing.T) { cases := []struct { name string host string port int - explicit bool wantHost string // kernelHostWithPort result for an accepted pair wantErr bool }{ - {"default 443 adds nothing", "h.databricks.com", 443, false, "h.databricks.com", false}, - {"unset port adds nothing", "h.databricks.com", 0, false, "h.databricks.com", false}, - {"custom port", "h.databricks.com", 8443, false, "h.databricks.com:8443", false}, - {"IPv4 literal", "127.0.0.1", 8443, false, "127.0.0.1:8443", false}, - {"bare IPv6 is bracketed", "::1", 8443, false, "[::1]:8443", false}, - {"bracketed IPv6", "[::1]", 8443, false, "[::1]:8443", false}, - {"port 1", "h.databricks.com", 1, false, "h.databricks.com:1", false}, - {"port 65535", "h.databricks.com", 65535, false, "h.databricks.com:65535", false}, - {"embedded port equal", "h.databricks.com:8443", 8443, true, "h.databricks.com:8443", false}, - {"embedded port equal, IPv6", "[::1]:8443", 8443, true, "[::1]:8443", false}, - {"unset 443 defers to embedded", "h.databricks.com:8443", 443, false, "h.databricks.com:8443", false}, - {"explicit 443 matches embedded", "h.databricks.com:443", 443, true, "h.databricks.com:443", false}, - {"explicit 443 differs from embedded: embedded wins", "h.databricks.com:8443", 443, true, "h.databricks.com:8443", false}, - {"embedded port differs: embedded wins", "h.databricks.com:7000", 8443, false, "h.databricks.com:7000", false}, - {"embedded port differs, IPv6: embedded wins", "[::1]:7000", 8443, false, "[::1]:7000", false}, - {"trailing slash on bare host", "h.databricks.com/", 8443, false, "h.databricks.com:8443", false}, - {"trailing slash on bare host, default port", "h.databricks.com/", 443, false, "h.databricks.com", false}, - {"trailing slash with embedded port, default port", "h.databricks.com:8443/", 443, false, "h.databricks.com:8443", false}, - {"trailing slash with embedded port, equal port", "h.databricks.com:8443/", 8443, true, "h.databricks.com:8443", false}, - {"trailing slash, IPv6", "[::1]/", 8443, false, "[::1]:8443", false}, - {"path after host", "h.databricks.com/x", 8443, false, "", true}, - {"path with embedded port and WithPort", "h.io/path:8443", 8443, true, "", true}, - {"path with embedded port, default port", "h.io/path:8443", 443, false, "", true}, - {"query before embedded port", "h.io?q=1:8443", 8443, true, "", true}, - {"explicit 443 with embedded port: embedded wins", "h.io:8443", 443, true, "h.io:8443", false}, - {"path after host, default port", "h.databricks.com/x", 443, false, "", true}, - {"query after host", "h.databricks.com?x", 8443, false, "", true}, - {"fragment after host", "h.databricks.com#x", 8443, false, "", true}, - {"negative port", "h.databricks.com", -1, false, "", true}, - {"port above range", "h.databricks.com", 65536, false, "", true}, - {"embedded port not a number", "h.databricks.com:notaport", 8443, false, "", true}, - {"embedded port out of range", "h.databricks.com:70000", 0, false, "", true}, - {"embedded port zero", "h.databricks.com:0", 0, false, "", true}, - {"unclosed IPv6 bracket", "[::1", 8443, false, "", true}, - {"text after IPv6 bracket", "[::1]x", 8443, false, "", true}, - {"stray closing bracket", "h.databricks.com]", 8443, false, "", true}, + {"default 443 adds nothing", "h.databricks.com", 443, "h.databricks.com", false}, + {"unset port adds nothing", "h.databricks.com", 0, "h.databricks.com", false}, + {"custom port", "h.databricks.com", 8443, "h.databricks.com:8443", false}, + {"IPv4 literal", "127.0.0.1", 8443, "127.0.0.1:8443", false}, + {"bare IPv6 is bracketed", "::1", 8443, "[::1]:8443", false}, + {"bracketed IPv6", "[::1]", 8443, "[::1]:8443", false}, + {"port 1", "h.databricks.com", 1, "h.databricks.com:1", false}, + {"port 65535", "h.databricks.com", 65535, "h.databricks.com:65535", false}, + {"embedded port equal", "h.databricks.com:8443", 8443, "h.databricks.com:8443", false}, + {"embedded port equal, IPv6", "[::1]:8443", 8443, "[::1]:8443", false}, + {"443 with embedded 443", "h.databricks.com:443", 443, "h.databricks.com:443", false}, + {"443 with embedded port: embedded wins", "h.databricks.com:8443", 443, "h.databricks.com:8443", false}, + {"embedded port differs: embedded wins", "h.databricks.com:7000", 8443, "h.databricks.com:7000", false}, + {"embedded port differs, IPv6: embedded wins", "[::1]:7000", 8443, "[::1]:7000", false}, + {"trailing slash on bare host", "h.databricks.com/", 8443, "h.databricks.com:8443", false}, + {"trailing slash on bare host, default port", "h.databricks.com/", 443, "h.databricks.com", false}, + {"trailing slash with embedded port, default port", "h.databricks.com:8443/", 443, "h.databricks.com:8443", false}, + {"trailing slash with embedded port, equal port", "h.databricks.com:8443/", 8443, "h.databricks.com:8443", false}, + {"trailing slash, IPv6", "[::1]/", 8443, "[::1]:8443", false}, + {"path, default port: unchanged", "h.databricks.com/x", 443, "h.databricks.com/x", false}, + {"path, unset port: unchanged", "h.databricks.com/x", 0, "h.databricks.com/x", false}, + {"path with colon, default port: unchanged", "h.io/path:8443", 443, "h.io/path:8443", false}, + {"path with custom port", "h.databricks.com/x", 8443, "", true}, + {"path with colon and custom port", "h.io/path:8443", 8443, "", true}, + {"query with colon and custom port", "h.io?q=1:8443", 8443, "", true}, + {"query with custom port", "h.databricks.com?x", 8443, "", true}, + {"fragment with custom port", "h.databricks.com#x", 8443, "", true}, + {"negative port", "h.databricks.com", -1, "", true}, + {"port above range", "h.databricks.com", 65536, "", true}, + {"embedded port not a number", "h.databricks.com:notaport", 8443, "", true}, + {"embedded port out of range", "h.databricks.com:70000", 0, "", true}, + {"embedded port zero", "h.databricks.com:0", 0, "", true}, + {"unclosed IPv6 bracket", "[::1", 8443, "", true}, + {"text after IPv6 bracket", "[::1]x", 8443, "", true}, + {"stray closing bracket", "h.databricks.com]", 8443, "", true}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { - err := validateKernelHostPort(tc.host, tc.port, tc.explicit) + err := validateKernelHostPort(tc.host, tc.port) if tc.wantErr { if !errors.Is(err, dbsqlerr.ErrInvalidKernelConfig) { t.Errorf("err = %v, want ErrInvalidKernelConfig", err) From c2bcd489ccf5f83d1f1ef4c73259b58c13634f9a Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Fri, 20 Nov 2026 01:56:30 +0000 Subject: [PATCH 7/7] kernel: parse the kernel host with net/url The kernel path folds WithPort into cfg.Host. The host is now parsed once per call with net/url, as the https authority the kernel connects to, instead of net.SplitHostPort plus hand-written bracket and path scans. Rules (unchanged unless noted): - A trailing "/" is trimmed first. With the default port (0 or 443) the host passes through unchanged. - A port embedded in the host wins over a differing WithPort, with a warning. An explicit 443, or 0, is the default and is never warned about. - WithPort outside 1..65535 is rejected. An embedded port that is empty, non-numeric or outside 1..65535 is rejected. - With a non-default port, a host with a path, query, fragment (including an empty "h#" or "h?") or user info is rejected. Unbalanced brackets are rejected. - Errors wrap ErrInvalidKernelConfig as before. Behavior changes from the reviewed revision (52bd5ac): - An IPv6 literal must be bracketed. A bare "::1" is rejected at any port (net/url cannot parse it). It never connected on main either: the kernel prepends https:// and its parse rejects it. - A host with a password ("user:pw@h") is rejected at any port. - A host net/url cannot parse (for example "h%zz" or a space) is rejected even at the default port, where it used to pass through to the kernel. The parse-once wrappers and the separate warning helper are gone: validateKernelHostPort logs the embedded-port warning, and kernelHostWithPort is the only fold, so buildKernelConfig and resolveKernelProxy keep their reviewed signatures. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 +- CONNECTION_PARAMETERS.md | 2 +- README.md | 2 +- kernel_config.go | 129 +++++++++++++++++++++------------------ kernel_config_test.go | 32 +++++++++- kernel_proxy_test.go | 6 +- 6 files changed, 104 insertions(+), 69 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d4ff273c..f25e63f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased (v2.0.0 prototype) -- Honor `WithPort` / DSN port on the kernel backend, including the automatic Reyden recovery onto the kernel: a non-default port is folded into the host (`host:port`, IPv6-safe) instead of being rejected. A port embedded in the host wins over a differing `WithPort` (a warning is logged), a zero port means the default, and a trailing `/` on the host is ignored. An out-of-range port, or a non-default port with a host that has a path, query or fragment, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). +- Honor `WithPort` / DSN port on the kernel backend, including the automatic Reyden recovery onto the kernel: a non-default port is folded into the host (`host:port`; an IPv6 literal must be bracketed, as `[::1]:8443`) instead of being rejected. A port embedded in the host wins over a differing `WithPort` (a warning is logged), a zero port means the default, and a trailing `/` on the host is ignored. An out-of-range port, a malformed or empty port in the host, or a non-default port with a host that has a path, query, fragment or credentials, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded). - Render non-finite FLOAT / DOUBLE values inside ARRAY / MAP / STRUCT results as the JSON strings `"NaN"`, `"Infinity"`, and `"-Infinity"` on both backends instead of failing row iteration. SQL NULL remains `null`; top-level floats remain numeric (databricks/databricks-sql-go#573). - **Behavior change:** Non-finite floating-point MAP keys use the same spellings as values on both backends. On the kernel backend, keys previously rendered as `"+Inf"` and `"-Inf"` now render as `"Infinity"` and `"-Infinity"`; `"NaN"` is unchanged. On Thrift, these keys previously failed row iteration (databricks/databricks-sql-go#573). - Fix the module-root package conflict when building, linting, or running Gazelle with the `tools` build tag (databricks/databricks-sql-go#162). diff --git a/CONNECTION_PARAMETERS.md b/CONNECTION_PARAMETERS.md index 3d09179b..18920a25 100644 --- a/CONNECTION_PARAMETERS.md +++ b/CONNECTION_PARAMETERS.md @@ -40,7 +40,7 @@ wire. They are listed under [Kernel client-result confs](#kernel-client-result-c |---|---|:---:|:---:|---|---| | *(host)* | `WithServerHostname` | ✅ | ✅ | *(required)* | Workspace hostname. | | *(path)* | `WithHTTPPath` | ✅ | ✅ | required on Thrift except localhost; required on kernel unless `warehouseId` is set | Warehouse/endpoint HTTP path. On the kernel path, a bare warehouse id can be used instead. | -| *(port)* | `WithPort` | ✅ | ✅ | `443` for connector options; required in a DSN | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged and the embedded port is used); an out-of-range port, or a non-default port with a host that has a path, query or fragment, is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | +| *(port)* | `WithPort` | ✅ | ✅ | `443` for connector options; required in a DSN | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged and the embedded port is used); an out-of-range port, a malformed port in the host, or a non-default port with a host that has a path, query, fragment or credentials, is rejected at connect. An IPv6 literal must be bracketed (`[::1]`, not `::1`). With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | | *(scheme)* | *(via `WithServerHostname`)* | ✅ | ⚠️ | `https` | Kernel accepts `https` only and rejects `http`. A bare `localhost` passed to `WithServerHostname` selects `http`; a scheme-less DSN, including localhost, is prefixed with `https`. | | `warehouseId` | `WithWarehouseID` | ⚠️ | ✅ | | Bare warehouse id. The kernel normally prefers it over the HTTP path; a canonical path carrying `?o=` takes precedence so workspace-org routing is preserved. The Thrift backend **silently ignores** it. | | `catalog` | `WithInitialNamespace` | ✅ | ✅ | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. | diff --git a/README.md b/README.md index ef83c70a..e187e97c 100644 --- a/README.md +++ b/README.md @@ -240,7 +240,7 @@ session parameter on both backends. |---|---|---|---|---| | *(host)* | `WithServerHostname` | Both | *(required)* | Workspace hostname. | | *(path)* | `WithHTTPPath` | Both | *(required)* | Warehouse/endpoint HTTP path. | -| *(port)* | `WithPort` | Both | `443` | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged); an out-of-range port, or a non-default port with a host that has a path, query or fragment, is rejected at connect. With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | +| *(port)* | `WithPort` | Both | `443` | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged); an out-of-range port, a malformed port in the host, or a non-default port with a host that has a path, query, fragment or credentials, is rejected at connect. An IPv6 literal must be bracketed (`[::1]`, not `::1`). With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. | | `warehouseId` | `WithWarehouseID` | SEA only | | Bare warehouse id; the kernel routes by it (preferred over the HTTP path). **The Thrift backend ignores it.** | | `catalog` | `WithInitialNamespace` | Both | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. | | `schema` | `WithInitialNamespace` | Both | | Initial schema. Kernel applies it post-connect via `USE SCHEMA`. | diff --git a/kernel_config.go b/kernel_config.go index d841d30b..b1c04f0d 100644 --- a/kernel_config.go +++ b/kernel_config.go @@ -85,7 +85,6 @@ func validateKernelConfigContext(ctx context.Context, cfg *config.Config) (kerne if err := validateKernelHostPort(cfg.Host, cfg.Port); err != nil { return kernel.Auth{}, err } - warnEmbeddedPortWins(cfg.Host, cfg.Port) // Transport (WithTransport, a custom http.RoundTripper carrying a custom CA // bundle / mTLS / proxy): the kernel uses its own Rust HTTP stack below the C // ABI and never sees a Go RoundTripper, so a custom Transport would be silently @@ -239,90 +238,98 @@ func buildKernelConfig(cfg *config.Config, kauth kernel.Auth) kernel.Config { return kc } -// unbalancedBrackets reports a host whose IPv6 brackets do not pair up: an unclosed `[`, text -// after the closing `]` other than a `:port`, or a stray bracket in a plain name. -func unbalancedBrackets(host string) bool { - if !strings.HasPrefix(host, "[") { - return strings.ContainsAny(host, "[]") - } - end := strings.IndexByte(host, ']') - return end < 0 || (end+1 < len(host) && host[end+1] != ':') +// kernelHost is cfg.Host split by net/url: the parts the kernel port logic needs. +type kernelHost struct { + raw string // cfg.Host with a trailing "/" trimmed; the kernel gets it unchanged when no port is folded + name string // host name without IPv6 brackets (url.URL.Hostname) + port string // digits after the authority's ":"; "" when there is none + hasPort bool // the authority has a ":", even with no digits after it ("h:") + hasPath bool // a path, query or fragment follows the host, or a bare "#" (url.URL drops an empty fragment) + userinfo bool // a user@ prefix is present + password bool // the user@ prefix carries a password } -// normaliseKernelHost trims trailing "/" the way the kernel's normalise_host does, so the port -// logic below sees the same authority the kernel connects to. -func normaliseKernelHost(host string) string { - return strings.TrimRight(host, "/") +// parseKernelHost parses cfg.Host (trailing "/" trimmed) as the authority of an https URL, which is the +// form the kernel connects to. The error is url.Parse's; the fields are set only when it is nil. +func parseKernelHost(host string) (kernelHost, error) { + h := kernelHost{raw: strings.TrimRight(host, "/")} + u, err := url.Parse("https://" + h.raw) + if err != nil { + return h, err + } + h.name = u.Hostname() + h.port = u.Port() + h.hasPort = h.port != "" || strings.HasSuffix(u.Host, ":") + h.hasPath = u.Path != "" || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" || strings.Contains(h.raw, "#") + h.userinfo = u.User != nil + _, h.password = u.User.Password() + return h, nil } -// validateKernelHostPort rejects an out-of-range WithPort, a non-default WithPort with a host that has -// a path, query or fragment, unbalanced IPv6 brackets, and an out-of-range or malformed port embedded -// in the host. WithDefaults fills 443 when no port is set. A port embedded in the host wins over -// WithPort (see warnEmbeddedPortWins), so a differing WithPort is not an error. +// validateKernelHostPort rejects an out-of-range WithPort, a host that net/url cannot parse (an IPv6 +// literal must be bracketed), and a non-default WithPort with a host that has a path, query, fragment or +// credentials. It also rejects unbalanced brackets and an empty, out-of-range or malformed port embedded +// in the host. WithDefaults fills 443 when no port is set. A port embedded in the host wins over a +// differing WithPort, with a warning rather than an error. func validateKernelHostPort(host string, port int) error { if port < 0 || port > 65535 { return fmt.Errorf("databricks: WithPort %d is out of range (want 1..65535, or 0 for the default): %w", port, dbsqlerr.ErrInvalidKernelConfig) } - host = normaliseKernelHost(host) + h, err := parseKernelHost(host) + if err != nil { + return fmt.Errorf("databricks: host %q is not a valid host (bracket an IPv6 literal): %v: %w", h.raw, err, dbsqlerr.ErrInvalidKernelConfig) + } + nonDefault := port != 0 && port != 443 // A path, query or fragment is checked before the embedded-port parse. With the default port the // host goes to the kernel unchanged, as before (the kernel appends its API paths after it). A - // non-default port cannot be folded into such a host, and was rejected before, so reject it. - if strings.ContainsAny(host, "/?#") { - if port != 0 && port != 443 { + // non-default port cannot be folded into such a host, so reject it. + if h.hasPath { + if nonDefault { return fmt.Errorf("databricks: host %q has a path, query or fragment, so WithPort %d cannot be applied: %w", - host, port, dbsqlerr.ErrInvalidKernelConfig) + h.raw, port, dbsqlerr.ErrInvalidKernelConfig) } return nil } - if unbalancedBrackets(host) { - return fmt.Errorf("databricks: host %q has unbalanced IPv6 brackets: %w", host, dbsqlerr.ErrInvalidKernelConfig) + // Credentials are kept only at the default port: a non-default port would have to be folded in without them. + if h.userinfo && nonDefault { + return fmt.Errorf("databricks: host %q carries credentials, so WithPort %d cannot be applied: %w", + h.raw, port, dbsqlerr.ErrInvalidKernelConfig) } - _, p, err := net.SplitHostPort(host) - if err != nil { - return nil // no embedded port + if h.password { + return fmt.Errorf("databricks: host %q carries a password: %w", h.raw, dbsqlerr.ErrInvalidKernelConfig) } - embedded, err := strconv.Atoi(p) - if err != nil || embedded < 1 || embedded > 65535 { - return fmt.Errorf("databricks: host %q carries an invalid port %q (want 1..65535): %w", host, p, dbsqlerr.ErrInvalidKernelConfig) + // url.Parse accepts a stray "]" in a plain name, so brackets are checked on the parsed host name + // (IPv6 brackets are already removed from it). + if strings.ContainsAny(h.name, "[]") { + return fmt.Errorf("databricks: host %q has unbalanced IPv6 brackets: %w", h.raw, dbsqlerr.ErrInvalidKernelConfig) } - return nil -} - -// warnEmbeddedPortWins logs when a non-default port (WithPort or the DSN) differs from the port embedded -// in the host. The embedded port is used, as on the kernel path before WithPort was honored. An -// explicit 443 is the default and is not warned about. -func warnEmbeddedPortWins(host string, port int) { - if port == 0 || port == 443 { - return + if !h.hasPort { + return nil // no embedded port } - _, p, err := net.SplitHostPort(normaliseKernelHost(host)) - if err != nil { - return + embedded, err := strconv.Atoi(h.port) + if err != nil || embedded < 1 || embedded > 65535 { + return fmt.Errorf("databricks: host %q carries an invalid port %q (want 1..65535): %w", h.raw, h.port, dbsqlerr.ErrInvalidKernelConfig) } - embedded, err := strconv.Atoi(p) - if err != nil || embedded == port { - return + // The embedded port is used, as on the kernel path before WithPort was honored. An explicit 443 is + // the default and is not warned about. + if nonDefault && embedded != port { + logger.Warn().Msgf("databricks: the host carries port %d, which differs from WithPort %d; connecting to port %d from the host", + embedded, port, embedded) } - logger.Warn().Msgf("databricks: the host carries port %d, which differs from WithPort %d; connecting to port %d from the host", - embedded, port, embedded) + return nil } -// kernelHostWithPort returns the Host the kernel connects to. The kernel has no port setter but -// keeps a `host:port` authority, so a non-default port is folded in (IPv6-safe). A trailing "/" is -// trimmed first. 0 and 443 add nothing, and a host that already carries a port is returned unchanged. +// kernelHostWithPort returns the Host the kernel connects to. The kernel has no port setter but keeps a +// `host:port` authority, so a non-default port is folded in with net.JoinHostPort, which brackets an IPv6 +// literal. A trailing "/" is trimmed first. 0 and 443 add nothing. A host that already carries a port, a +// path, query or fragment, or credentials is returned unchanged; validateKernelHostPort rejects the last +// two with a non-default port before the kernel is built. func kernelHostWithPort(host string, port int) string { - host = normaliseKernelHost(host) - if port == 0 || port == 443 { - return host - } - if _, _, err := net.SplitHostPort(host); err == nil { - return host - } - name := host - if strings.HasPrefix(host, "[") && strings.HasSuffix(host, "]") { - name = host[1 : len(host)-1] + h, err := parseKernelHost(host) + if err != nil || port == 0 || port == 443 || h.hasPort || h.hasPath || h.userinfo { + return h.raw } - return net.JoinHostPort(name, strconv.Itoa(port)) + return net.JoinHostPort(h.name, strconv.Itoa(port)) } func kernelTelemetryConfig(cfg *config.Config) *kernel.TelemetryConfig { diff --git a/kernel_config_test.go b/kernel_config_test.go index 0926fb4b..75f1a3e4 100644 --- a/kernel_config_test.go +++ b/kernel_config_test.go @@ -975,7 +975,8 @@ func TestKernelRetryConfig(t *testing.T) { // TestKernelCustomPort covers WithPort on the kernel backend: a non-default port is folded into the // kernel Host, 0 and 443 add nothing, a port embedded in the host wins, a trailing "/" is trimmed, // and a host with a path passes through unchanged unless a non-default port would be folded into it. -// An out-of-range port or unbalanced IPv6 brackets are rejected with ErrInvalidKernelConfig. +// Host parsing uses net/url. An out-of-range port, unbalanced IPv6 brackets, or a bare (unbracketed) +// IPv6 literal are rejected with ErrInvalidKernelConfig, so an IPv6 literal must be bracketed. func TestKernelCustomPort(t *testing.T) { cases := []struct { name string @@ -988,7 +989,7 @@ func TestKernelCustomPort(t *testing.T) { {"unset port adds nothing", "h.databricks.com", 0, "h.databricks.com", false}, {"custom port", "h.databricks.com", 8443, "h.databricks.com:8443", false}, {"IPv4 literal", "127.0.0.1", 8443, "127.0.0.1:8443", false}, - {"bare IPv6 is bracketed", "::1", 8443, "[::1]:8443", false}, + {"bare IPv6 is rejected: an IPv6 literal must be bracketed", "::1", 8443, "", true}, {"bracketed IPv6", "[::1]", 8443, "[::1]:8443", false}, {"port 1", "h.databricks.com", 1, "h.databricks.com:1", false}, {"port 65535", "h.databricks.com", 65535, "h.databricks.com:65535", false}, @@ -1019,6 +1020,33 @@ func TestKernelCustomPort(t *testing.T) { {"unclosed IPv6 bracket", "[::1", 8443, "", true}, {"text after IPv6 bracket", "[::1]x", 8443, "", true}, {"stray closing bracket", "h.databricks.com]", 8443, "", true}, + {"stray closing bracket, default port", "h.databricks.com]", 443, "", true}, + {"text after IPv6 bracket, default port", "[::1]x", 443, "", true}, + {"unclosed IPv6 bracket, default port", "[::1", 443, "", true}, + // net/url rules: an empty port, credentials, an empty fragment and bare IPv6 literals. + {"empty port after colon", "h:", 8443, "", true}, + {"empty port after colon, default port", "h:", 443, "", true}, + {"bare IPv6, default port: rejected (bracket it)", "::1", 443, "", true}, + {"bare IPv6 with zone is rejected", "fe80::1%eth0", 8443, "", true}, + {"userinfo, default port: unchanged", "user@h", 443, "user@h", false}, + {"userinfo, custom port: rejected", "user@h", 8443, "", true}, + {"userinfo with an embedded port, custom port: rejected", "user@h:8443", 9443, "", true}, + {"userinfo with an embedded port, default port: unchanged", "user@h:8443", 443, "user@h:8443", false}, + {"password, default port: rejected", "user:pw@h", 443, "", true}, + {"password, custom port: rejected", "user:pw@h", 8443, "", true}, + {"empty fragment, custom port: rejected (h#)", "h#", 8443, "", true}, + {"empty fragment, default port: unchanged (h#)", "h#", 443, "h#", false}, + {"embedded port out of range (h:70000)", "h:70000", 8443, "", true}, + {"embedded port not a number (h:abc)", "h:abc", 8443, "", true}, + {"mixed-case host is kept (custom port)", "H.Example.COM", 8443, "H.Example.COM:8443", false}, + {"mixed-case host is kept (default port)", "H.Example.COM", 443, "H.Example.COM", false}, + {"trailing slash after embedded port", "h:8443/", 8443, "h:8443", false}, + {"path with custom port (h/x)", "h/x", 8443, "", true}, + {"path, default port: unchanged (h/x)", "h/x", 443, "h/x", false}, + {"query with custom port (h?q)", "h?q", 8443, "", true}, + {"query, default port: unchanged (h?q)", "h?q", 443, "h?q", false}, + {"fragment with custom port (h#f)", "h#f", 8443, "", true}, + {"fragment, default port: unchanged (h#f)", "h#f", 443, "h#f", false}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { diff --git a/kernel_proxy_test.go b/kernel_proxy_test.go index 9316e470..4890c742 100644 --- a/kernel_proxy_test.go +++ b/kernel_proxy_test.go @@ -68,12 +68,12 @@ func TestProxyForEndpoint(t *testing.T) { want: "http://corp-proxy:3128", }, { - // An unbracketed IPv6 literal with a port must be bracketed in the endpoint, + // A bracketed IPv6 literal with a port is folded into the endpoint as [addr]:port, // the same way the kernel Host is built. - name: "IPv6 literal with Port is bracketed", + name: "bracketed IPv6 literal with Port is folded", cfg: func() *config.Config { c := config.WithDefaults() - c.Host = "::1" + c.Host = "[::1]" c.Port = 8443 c.HTTPPath = "/sql/1.0/warehouses/abc" return c