Skip to content
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## Unreleased (v2.0.0 prototype)

- Honor `WithPort` / DSN port on the kernel backend, including the automatic Reyden recovery onto the kernel: a non-default port is folded into the host (`host:port`; an IPv6 literal must be bracketed, as `[::1]:8443`) instead of being rejected. A port embedded in the host wins over a differing `WithPort` (a warning is logged), a zero port means the default, and a trailing `/` on the host is ignored. An out-of-range port, a malformed or empty port in the host, or a non-default port with a host that has a path, query, fragment or credentials, is rejected with `ErrInvalidKernelConfig`. With OAuth and a non-default port, the kernel uses the same `host:port` for OAuth; Thrift keeps OAuth on the bare host (documented, not guarded).
- Render non-finite FLOAT / DOUBLE values inside ARRAY / MAP / STRUCT results as the JSON strings `"NaN"`, `"Infinity"`, and `"-Infinity"` on both backends instead of failing row iteration. SQL NULL remains `null`; top-level floats remain numeric (databricks/databricks-sql-go#573).
- **Behavior change:** Non-finite floating-point MAP keys use the same spellings as values on both backends. On the kernel backend, keys previously rendered as `"+Inf"` and `"-Inf"` now render as `"Infinity"` and `"-Infinity"`; `"NaN"` is unchanged. On Thrift, these keys previously failed row iteration (databricks/databricks-sql-go#573).
- Fix the module-root package conflict when building, linting, or running Gazelle with the `tools` build tag (databricks/databricks-sql-go#162).
Expand Down
2 changes: 1 addition & 1 deletion CONNECTION_PARAMETERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ wire. They are listed under [Kernel client-result confs](#kernel-client-result-c
|---|---|:---:|:---:|---|---|
| *(host)* | `WithServerHostname` | ✅ | ✅ | *(required)* | Workspace hostname. |
| *(path)* | `WithHTTPPath` | ✅ | ✅ | required on Thrift except localhost; required on kernel unless `warehouseId` is set | Warehouse/endpoint HTTP path. On the kernel path, a bare warehouse id can be used instead. |
| *(port)* | `WithPort` | ✅ | ⚠️ | `443` for connector options; required in a DSN | Kernel accepts **443 only** and rejects any other port. |
| *(port)* | `WithPort` | ✅ | ✅ | `443` for connector options; required in a DSN | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged and the embedded port is used); an out-of-range port, a malformed port in the host, or a non-default port with a host that has a path, query, fragment or credentials, is rejected at connect. An IPv6 literal must be bracketed (`[::1]`, not `::1`). With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. |
| *(scheme)* | *(via `WithServerHostname`)* | ✅ | ⚠️ | `https` | Kernel accepts `https` only and rejects `http`. A bare `localhost` passed to `WithServerHostname` selects `http`; a scheme-less DSN, including localhost, is prefixed with `https`. |
| `warehouseId` | `WithWarehouseID` | ⚠️ | ✅ | | Bare warehouse id. The kernel normally prefers it over the HTTP path; a canonical path carrying `?o=` takes precedence so workspace-org routing is preserved. The Thrift backend **silently ignores** it. |
| `catalog` | `WithInitialNamespace` | ✅ | ✅ | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. |
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ session parameter on both backends.
|---|---|---|---|---|
| *(host)* | `WithServerHostname` | Both | *(required)* | Workspace hostname. |
| *(path)* | `WithHTTPPath` | Both | *(required)* | Warehouse/endpoint HTTP path. |
| *(port)* | `WithPort` | Thrift only | `443` | Kernel connects on **443 only** and rejects any other port. |
| *(port)* | `WithPort` | Both | `443` | Kernel folds a non-default port into the host it connects to. A port embedded in the host wins over a differing `WithPort` (a warning is logged); an out-of-range port, a malformed port in the host, or a non-default port with a host that has a path, query, fragment or credentials, is rejected at connect. An IPv6 literal must be bracketed (`[::1]`, not `::1`). With OAuth the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare host. |
| `warehouseId` | `WithWarehouseID` | SEA only | | Bare warehouse id; the kernel routes by it (preferred over the HTTP path). **The Thrift backend ignores it.** |
| `catalog` | `WithInitialNamespace` | Both | | Initial catalog. Kernel applies it post-connect via `USE CATALOG`. |
| `schema` | `WithInitialNamespace` | Both | | Initial schema. Kernel applies it post-connect via `USE SCHEMA`. |
Expand Down
23 changes: 23 additions & 0 deletions connector_reyden_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,29 @@ func TestReydenReactiveRecovery(t *testing.T) {
assert.True(t, warehouse_cache.IsKnownReyden(tt.host, tt.warehouseID),
"warehouse should be marked Reyden after the rejection")
})

t.Run("recovery onto the kernel keeps a custom WithPort", func(t *testing.T) {
defer warehouse_cache.ClearCache()

tt := NewTestReydenFallback()
var kernelHost string
conn := tt.makeConnector(
func(ctx context.Context, cfg *config.Config, client *http.Client) (backend.Backend, error) {
return &fakeThriftBackend{openSessionErr: dbsqlerrint.NewReydenThriftUnsupportedError(
"Lakehouse/RT is not supported for Thrift protocol")}, nil
},
func(ctx context.Context, cfg *config.Config) (backend.Backend, error) {
require.NoError(t, validateKernelHostPort(cfg.Host, cfg.Port))
kernelHost = buildKernelConfig(cfg, kernel.Auth{Mode: kernel.AuthPAT, Token: "test-token"}).Host
return &fakeKernelBackend{}, nil
},
)
WithPort(8443)(conn.cfg)

_, _, err := conn.openSessionWithReydenFallback(context.Background())
require.NoError(t, err)
assert.Equal(t, "reyden.example.com:8443", kernelHost)
})
}

func TestReydenPreCheck(t *testing.T) {
Expand Down
32 changes: 32 additions & 0 deletions internal/config/thrift_endpoint_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package config

import "testing"

// TestThriftEndpointUnchanged pins the Thrift endpoint for the port inputs the kernel custom-port
// change decides on. Thrift is not changed by that work, so these strings are what it builds today.
// An embedded host port plus the default 443 yields an unparseable URL on Thrift; that is the baseline.
func TestThriftEndpointUnchanged(t *testing.T) {
cases := []struct {
name string
host string
port int
want string
}{
{"bare host, default port", "h.io", 443, "https://h.io:443/sql/1.0/warehouses/x"},
{"bare host, WithPort", "h.io", 8443, "https://h.io:8443/sql/1.0/warehouses/x"},
{"embedded port, default port", "h.io:8443", 443, "https://h.io:8443:443/sql/1.0/warehouses/x"},
{"embedded port, WithPort", "h.io:8443", 8443, "https://h.io:8443:8443/sql/1.0/warehouses/x"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
c := &Config{Protocol: "https", Host: tc.host, Port: tc.port, HTTPPath: "/sql/1.0/warehouses/x"}
got, err := c.ToEndpointURL()
if err != nil {
t.Fatalf("unexpected err %v", err)
}
if got != tc.want {
t.Errorf("ToEndpointURL() = %q, want %q", got, tc.want)
}
})
}
}
111 changes: 102 additions & 9 deletions kernel_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"net"
"net/url"
"strconv"
"strings"
Expand All @@ -15,6 +16,7 @@ import (
"github.com/databricks/databricks-sql-go/internal/backend/kernel"
"github.com/databricks/databricks-sql-go/internal/client"
"github.com/databricks/databricks-sql-go/internal/config"
"github.com/databricks/databricks-sql-go/logger"
"github.com/databricks/databricks-sql-go/telemetry"
)

Expand Down Expand Up @@ -73,18 +75,15 @@ func validateKernelConfigContext(ctx context.Context, cfg *config.Config) (kerne
// folds its server conf (spark.sql.thriftserver.metadata.metricview.enabled=true)
// into SessionConf backend-neutrally, so the kernel path sends the identical conf
// the Thrift path does. No per-backend handling needed here.
// Port / Protocol: the kernel C ABI takes only a bare host and connects over
// https:443; it has no port or scheme setter. The Thrift path honors a custom
// port/scheme via ToEndpointURL, so a non-default value here would be silently
// ignored on the kernel path (it would just hit 443) — reject it instead, per
// the "nothing silently ignored" contract. Defaults (https/443) are fine.
// Protocol: the kernel always connects over https, so a non-https value would be ignored.
if cfg.Protocol != "" && cfg.Protocol != "https" {
return kernel.Auth{}, fmt.Errorf("databricks: a non-https protocol is %w "+
"(it connects over https); use the default (Thrift) backend", dbsqlerr.ErrNotSupportedByKernel)
}
if cfg.Port != 0 && cfg.Port != 443 {
return kernel.Auth{}, fmt.Errorf("databricks: a non-default port (WithPort) is %w "+
"(it connects on 443); omit it or use the default (Thrift) backend", dbsqlerr.ErrNotSupportedByKernel)
// Port: the kernel folds host:port (see kernelHostWithPort); reject a bad port or host here. An
// embedded host port wins over a differing WithPort, with a warning rather than an error.
if err := validateKernelHostPort(cfg.Host, cfg.Port); err != nil {
return kernel.Auth{}, err
}
// Transport (WithTransport, a custom http.RoundTripper carrying a custom CA
// bundle / mTLS / proxy): the kernel uses its own Rust HTTP stack below the C
Expand Down Expand Up @@ -155,7 +154,7 @@ func validateKernelConfigContext(ctx context.Context, cfg *config.Config) (kerne
// with newKernelBackend's kernel.Config assembly.
func buildKernelConfig(cfg *config.Config, kauth kernel.Auth) kernel.Config {
kc := kernel.Config{
Host: cfg.Host,
Host: kernelHostWithPort(cfg.Host, cfg.Port),
Comment thread
peco-review-bot[bot] marked this conversation as resolved.
Resolved
HTTPPath: cfg.HTTPPath,
WarehouseID: cfg.WarehouseID,
Auth: kauth,
Expand Down Expand Up @@ -239,6 +238,100 @@ func buildKernelConfig(cfg *config.Config, kauth kernel.Auth) kernel.Config {
return kc
}

// kernelHost is cfg.Host split by net/url: the parts the kernel port logic needs.
type kernelHost struct {
raw string // cfg.Host with a trailing "/" trimmed; the kernel gets it unchanged when no port is folded
name string // host name without IPv6 brackets (url.URL.Hostname)
port string // digits after the authority's ":"; "" when there is none
hasPort bool // the authority has a ":", even with no digits after it ("h:")
hasPath bool // a path, query or fragment follows the host, or a bare "#" (url.URL drops an empty fragment)
userinfo bool // a user@ prefix is present
password bool // the user@ prefix carries a password
}

// parseKernelHost parses cfg.Host (trailing "/" trimmed) as the authority of an https URL, which is the
// form the kernel connects to. The error is url.Parse's; the fields are set only when it is nil.
func parseKernelHost(host string) (kernelHost, error) {
h := kernelHost{raw: strings.TrimRight(host, "/")}
u, err := url.Parse("https://" + h.raw)
if err != nil {
return h, err
}
h.name = u.Hostname()
h.port = u.Port()
h.hasPort = h.port != "" || strings.HasSuffix(u.Host, ":")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — net/url accepts https://::1 and reads the trailing 1 as a port, so a bare IPv6 host passes validation and WithPort is ignored. ParseDSN also strips brackets (ucfg.Host = parsedURL.Hostname()), so DSN IPv6 hosts reach this path unbracketed. Detect unbracketed IPv6 explicitly and re-bracket DSN-derived hosts.

🔀 Found by GPT-6.1 Sol · confirmed by the judge

h.hasPath = u.Path != "" || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" || strings.Contains(h.raw, "#")
h.userinfo = u.User != nil
_, h.password = u.User.Password()
return h, nil
}

// validateKernelHostPort rejects an out-of-range WithPort, a host that net/url cannot parse (an IPv6
// literal must be bracketed), and a non-default WithPort with a host that has a path, query, fragment or
// credentials. It also rejects unbalanced brackets and an empty, out-of-range or malformed port embedded
// in the host. WithDefaults fills 443 when no port is set. A port embedded in the host wins over a
// differing WithPort, with a warning rather than an error.
func validateKernelHostPort(host string, port int) error {
if port < 0 || port > 65535 {
return fmt.Errorf("databricks: WithPort %d is out of range (want 1..65535, or 0 for the default): %w", port, dbsqlerr.ErrInvalidKernelConfig)
}
h, err := parseKernelHost(host)
if err != nil {
return fmt.Errorf("databricks: host %q is not a valid host (bracket an IPv6 literal): %v: %w", h.raw, err, dbsqlerr.ErrInvalidKernelConfig)
}
nonDefault := port != 0 && port != 443
// A path, query or fragment is checked before the embedded-port parse. With the default port the
// host goes to the kernel unchanged, as before (the kernel appends its API paths after it). A
// non-default port cannot be folded into such a host, so reject it.
if h.hasPath {
if nonDefault {
return fmt.Errorf("databricks: host %q has a path, query or fragment, so WithPort %d cannot be applied: %w",
h.raw, port, dbsqlerr.ErrInvalidKernelConfig)
}
return nil
}
// Credentials are kept only at the default port: a non-default port would have to be folded in without them.
if h.userinfo && nonDefault {
return fmt.Errorf("databricks: host %q carries credentials, so WithPort %d cannot be applied: %w",
h.raw, port, dbsqlerr.ErrInvalidKernelConfig)
}
if h.password {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — A host with a password is now rejected at every port, but the CHANGELOG and README only describe rejecting credentials with a non-default port. Document the rejection at all ports, or drop it.

🔀 Found by Claude Opus 5.5 · confirmed by the judge

return fmt.Errorf("databricks: host %q carries a password: %w", h.raw, dbsqlerr.ErrInvalidKernelConfig)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium — The credential and parse-error messages format the raw host, which includes the userinfo and password, into caller-visible errors. Format only the host name, or strip the userinfo, before building these errors.

🔀 Found by both reviewers · confirmed by the judge

}
// url.Parse accepts a stray "]" in a plain name, so brackets are checked on the parsed host name
// (IPv6 brackets are already removed from it).
if strings.ContainsAny(h.name, "[]") {
return fmt.Errorf("databricks: host %q has unbalanced IPv6 brackets: %w", h.raw, dbsqlerr.ErrInvalidKernelConfig)
}
if !h.hasPort {
return nil // no embedded port
}
embedded, err := strconv.Atoi(h.port)
if err != nil || embedded < 1 || embedded > 65535 {
return fmt.Errorf("databricks: host %q carries an invalid port %q (want 1..65535): %w", h.raw, h.port, dbsqlerr.ErrInvalidKernelConfig)
}
// The embedded port is used, as on the kernel path before WithPort was honored. An explicit 443 is
// the default and is not warned about.
if nonDefault && embedded != port {
logger.Warn().Msgf("databricks: the host carries port %d, which differs from WithPort %d; connecting to port %d from the host",
embedded, port, embedded)
}
return nil
}

// kernelHostWithPort returns the Host the kernel connects to. The kernel has no port setter but keeps a
// `host:port` authority, so a non-default port is folded in with net.JoinHostPort, which brackets an IPv6
// literal. A trailing "/" is trimmed first. 0 and 443 add nothing. A host that already carries a port, a
// path, query or fragment, or credentials is returned unchanged; validateKernelHostPort rejects the last
// two with a non-default port before the kernel is built.
func kernelHostWithPort(host string, port int) string {
h, err := parseKernelHost(host)
if err != nil || port == 0 || port == 443 || h.hasPort || h.hasPath || h.userinfo {
return h.raw
}
return net.JoinHostPort(h.name, strconv.Itoa(port))
}

func kernelTelemetryConfig(cfg *config.Config) *kernel.TelemetryConfig {
var enabled *bool
if val, isSet := cfg.EnableTelemetry.Get(); isSet {
Expand Down
Loading
Loading