Repository navigation
Conversation
The kernel backend rejected any WithPort other than 443. The kernel has no port setter, but it keeps a `host:port` authority in its host, so the connector now folds a non-default port into the host it passes to the kernel. - kernelHostWithPort adds a non-default port with net.JoinHostPort (IPv6-safe). The default 443 and an unset port add nothing. - validateKernelHostPort rejects an out-of-range port, a host with unbalanced IPv6 brackets (for example `[::1` or `h.databricks.com]`), and a supplied port that disagrees with a port embedded in the host. An unset port (WithDefaults fills 443) defers to the embedded port. An explicit WithPort(443) or DSN port is supplied, so it must match an embedded port too (AC-P3: never drop a port). - UserConfig gains an internal PortExplicit flag, set by WithPort and the DSN parser. It is not a public option: internal/ is not importable by users. - Non-https protocols and the other kernel rejections are unchanged. With OAuth (M2M and U2M) and a non-default port, the kernel reaches the OAuth endpoints at the same host:port as the connection. Thrift keeps OAuth on the bare host. This is documented in CONNECTION_PARAMETERS.md, the README and the CHANGELOG, and is not guarded. Tests: one table-driven TestKernelCustomPort covers the host mapping, the validation (accept and reject, including explicit 443 and unbalanced brackets), and the kernel Host. The config, connector and driver tests expect PortExplicit where WithPort or the DSN set the port. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Medium
Looks good: one low concern. The port folding, the validation and the PortExplicit plumbing check out, including DeepCopy and the DSN path. Nit: the PortExplicit disposition is listed as "forwarded", but it is only consumed by validation.
This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.
proxyForEndpointFunc appended the port to cfg.Host even when the host already carried one, so host:8443 with Port 8443 built an invalid endpoint, the proxy was dropped and HTTPS_PROXY was ignored. It now uses the same authority as kernelHostWithPort, which leaves an embedded port alone and brackets IPv6 literals. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Nit
Looks good, with 1 low concern. The port folding and validation are correct for IPv4, IPv6 and embedded-port cases, and PortExplicit is set by both entry points (WithPort and the DSN parser) and kept by DeepCopy. Nit: the out-of-range error message says "want 1..65535", but port 0 is accepted as unset.
Other findings
- ⚪ Nit — The Reyden auto-recovery path (openSessionWithReydenFallback → getKernelBackend → buildKernelConfig) now folds a custom port into the kernel Host, where it was rejected before. That is reasonable, but no test or CHANGELOG line covers the recovery path. Add a recovery test with WithPort(8443) that asserts the kernel Host, and mention recovery in the CHANGELOG entry. (Found by Claude Opus 5.5 · confirmed by the judge)
Address the review nits on the kernel custom-port change: - The out-of-range WithPort error now says 0 is accepted as the default. - PortExplicit is classified as "validated", not "forwarded": it is consumed by validateKernelHostPort and never sent to the kernel. - A Reyden recovery test with WithPort(8443) asserts that the kernel Host is host:8443 after the Thrift KP001 rejection. The CHANGELOG entry mentions the recovery path. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
|
Fixed the review nits in 263c741:
Automated reply from the driver feature factory (Isaac). Not reviewed by a human. |
…hPort
Fold a trailing "/" in the host before any port handling, matching the
kernel's normalise_host, so WithServerHostname("https://h/") plus
WithPort(8443) connects to h:8443 instead of h/:8443, and
"https://h:8443/" keeps working.
Port precedence now matches the Node and Python kernel paths:
- A port embedded in the host wins over a differing WithPort. The driver
logs a warning naming both ports and connects to the embedded port.
- A zero (unset) WithPort means the default, with no :443 added.
- A host with a path, query or fragment after the trimmed "/" is
rejected with ErrInvalidKernelConfig. Base also rejected it: the kernel
builds {host}/api/2.0/sql, so h/x sent requests to /x/api/2.0/sql.
- Out-of-range WithPort and out-of-range or malformed embedded ports stay
rejected. Thrift also fails on them: an embedded port plus the default
443 yields an unparseable https://h:8443:443 URL, -1 is an invalid port,
and 70000 fails to dial.
Base behavior (origin/main): validateKernelConfigContext rejected any
WithPort other than 0 or 443 with ErrNotSupportedByKernel, and passed the
host through unchanged, so an embedded port connected to that port.
Tests: table cases for h/, h:8443/, h/x, [::1]/, embedded-vs-WithPort
(warn, embedded wins), explicit 443 against an embedded port, and an
invalid port (still rejected). Proxy endpoint test for a trailing-slash
host. Kernel loopback probe: embedded port dialed, explicit port never.
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
…xplicit 443
- A host with a path or query is rejected before the embedded-port parse, so
"h.io/path:8443" with an explicit port cannot slip through. The old code passed
the host verbatim to the kernel, which builds {host}/api/2.0/sql, so these
configs never connected.
- The differing-port warning is skipped for an explicit 443 (the default).
- Thrift is unchanged. TestThriftEndpointUnchanged pins its current endpoint
strings for the same port inputs, including the embedded-port-plus-443 case.
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
- A host with a path, query or fragment is now rejected only when a
non-default WithPort would have to be folded into it. The check still runs
after the trailing "/" trim and before the embedded-port parse. With no port
or 443 the host goes to the kernel unchanged, as on the base branch, where
the kernel keeps a path prefix ({host}/api/2.0/sql). The base branch rejected
every non-default WithPort, so nothing that worked before is rejected now.
- Remove the internal PortExplicit field. Once an explicit 443 stopped
warning, nothing read it, so WithPort, ParseDSN, DeepCopy and their tests
are back to the base branch.
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Resolve the CHANGELOG.md conflict by keeping both Unreleased entries: this PR's kernel port entry and main's non-finite float rendering entries. No code conflicts. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
The kernel path folds WithPort into cfg.Host. The host is now parsed once per call with net/url, as the https authority the kernel connects to, instead of net.SplitHostPort plus hand-written bracket and path scans. Rules (unchanged unless noted): - A trailing "/" is trimmed first. With the default port (0 or 443) the host passes through unchanged. - A port embedded in the host wins over a differing WithPort, with a warning. An explicit 443, or 0, is the default and is never warned about. - WithPort outside 1..65535 is rejected. An embedded port that is empty, non-numeric or outside 1..65535 is rejected. - With a non-default port, a host with a path, query, fragment (including an empty "h#" or "h?") or user info is rejected. Unbalanced brackets are rejected. - Errors wrap ErrInvalidKernelConfig as before. Behavior changes from the reviewed revision (52bd5ac): - An IPv6 literal must be bracketed. A bare "::1" is rejected at any port (net/url cannot parse it). It never connected on main either: the kernel prepends https:// and its parse rejects it. - A host with a password ("user:pw@h") is rejected at any port. - A host net/url cannot parse (for example "h%zz" or a space) is rejected even at the default port, where it used to pass through to the kernel. The parse-once wrappers and the separate warning helper are gone: validateKernelHostPort logs the embedded-port warning, and kernelHostWithPort is the only fold, so buildKernelConfig and resolveKernelProxy keep their reviewed signatures. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 2 Medium · 1 Low
Mostly solid, with 1 medium and 1 low concern. The port folding and validation are well tested, and the Reyden recovery and production paths both go through newKernelBackend's validation. The main problem is that a password in the host can end up in error messages.
This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.
| } | ||
| h.name = u.Hostname() | ||
| h.port = u.Port() | ||
| h.hasPort = h.port != "" || strings.HasSuffix(u.Host, ":") |
There was a problem hiding this comment.
🟡 Medium — net/url accepts https://::1 and reads the trailing 1 as a port, so a bare IPv6 host passes validation and WithPort is ignored. ParseDSN also strips brackets (ucfg.Host = parsedURL.Hostname()), so DSN IPv6 hosts reach this path unbracketed. Detect unbracketed IPv6 explicitly and re-bracket DSN-derived hosts.
🔀 Found by GPT-6.1 Sol · confirmed by the judge
| h.raw, port, dbsqlerr.ErrInvalidKernelConfig) | ||
| } | ||
| if h.password { | ||
| return fmt.Errorf("databricks: host %q carries a password: %w", h.raw, dbsqlerr.ErrInvalidKernelConfig) |
There was a problem hiding this comment.
🟡 Medium — The credential and parse-error messages format the raw host, which includes the userinfo and password, into caller-visible errors. Format only the host name, or strip the userinfo, before building these errors.
🔀 Found by both reviewers · confirmed by the judge
| return fmt.Errorf("databricks: host %q carries credentials, so WithPort %d cannot be applied: %w", | ||
| h.raw, port, dbsqlerr.ErrInvalidKernelConfig) | ||
| } | ||
| if h.password { |
There was a problem hiding this comment.
🔵 Low — A host with a password is now rejected at every port, but the CHANGELOG and README only describe rejecting credentials with a non-default port. Document the rejection at all ports, or drop it.
🔀 Found by Claude Opus 5.5 · confirmed by the judge
What type of PR is this?
Description
On the kernel backend,
WithPort(and the DSNport) is now honored. The kernel has no port setter, but it keeps ahost:portauthority in its host, so the connector folds a non-default port into the host it connects to.This revision parses the host with Go's standard
net/urlinstead ofnet.SplitHostPortplus hand-written bracket and path scans. The host is parsed once per call as thehttpsauthority the kernel connects to. The parse-once wrappers are gone, andbuildKernelConfig,resolveKernelProxyand the tests call the same functions as before.IPv6 literals must be bracketed.
[::1]works, and[::1]:8443works. A bare::1is rejected withErrInvalidKernelConfig. It never connected onmaineither, because the kernel prependshttps://and its URL parse rejects an unbracketed literal.Port and host rules:
WithPortor DSN port. The connector logs a warning naming both ports and connects to the embedded port. It does not return an error. An explicit 443 is the default and is never warned about.:443is added. With the default port (0 or 443) the host passes through unchanged./is trimmed first, matching the kernel'snormalise_host.WithServerHostname("https://h/") + WithPort(8443)connects toh:8443.WithPort(outside 1..65535) is rejected. An embedded port that is empty, non-numeric, or outside 1..65535 is rejected.h#orh?. A host with user info (user@h) is rejected, and so are unbalanced brackets.Everything else is unchanged: the non-https protocol rejection,
WithTimeout,WithTransport, the other kernel rejections, and the Thrift path. Errors still wrap the sameErrInvalidKernelConfigsentinel.Differences from the reviewed revision (52bd5ac)
::1) is rejected at any port. Before, it was accepted at the default port and folded to[::1]:8443with a custom port.user:pw@h) is rejected at any port. Before, it was rejected too, because itsSplitHostPortparse failed.net/urlcannot parse (for exampleh%zzorh i) is rejected even at the default port, withErrInvalidKernelConfig. Before, it passed through unchanged, and the kernel's WHATWG host parse rejects it, so it never connected. The failure moves earlier and gets a clearer error; no working connection is lost. A bracketed IPv6 host is unaffected.]in a plain host name is still rejected.url.Parseaccepts it, so the bracket check stays explicit.Net change in the kernel files versus 52bd5ac:
kernel_config.go+68 / -61,kernel_config_test.go+30 / -2,kernel_proxy_test.go+3 / -3, and one-line edits to the CHANGELOG,CONNECTION_PARAMETERS.mdandREADME.md.kernel_proxy.gois unchanged from 52bd5ac.Documented, not guarded: with OAuth (M2M or U2M) and a non-default port, the kernel reaches the OAuth endpoints at the same
host:portas the connection. Thrift keeps OAuth on the bare host. Noted inCONNECTION_PARAMETERS.md,README.mdand the CHANGELOG.How is this tested?
Unit tests
E2E Tests
Manually
N/A
TestKernelCustomPortis table-driven. It covers the rules above, the bare-IPv6 rejection, and theh#,h?,user@handuser:pw@hcases, with the default and a custom port.TestKernelCustomPortalso covers theWithServerHostname("https://h/") + WithPort(8443)fold, and an embedded port with a trailing/. The proxy endpoint test covers a bracketed IPv6 literal with a port.CGO_ENABLED=0 go test -count=1 ./...passes.go vet,gofmt -sand the repo's golangci-lint config (v2.15.0, default build) report no issues.Kernel-tagged suite (
-tags databricks_kernel, bindings v1.4.0): every package passes exceptTestKernelCloudFetchDisabledPreservesUnsupportedServerError. That test fails the same way at 52bd5ac, so it is not caused by this change.Live, kernel and Thrift paths, DBSQL and Reyden warehouses, at this commit. See the Live section below.
Live
Probe built from this commit. Each case runs one connect plus
SELECT 1, and the effective backend is checked. Warm samples are in milliseconds.WithPort(443)*kernel.KernelBackend, warm 570/263/235:443,WithPort(0):443,WithPort(8443)(embedded wins, warning logged)WithPort(443)*thrift.Backend, warm 177/175/169The first Reyden attempt of two cases hit the 100 s wrapper deadline during a slow spell, with token-federation fallback logged. Both passed on a single retry.
No 52bd5ac live run was made, so these are not compared against the reviewed revision. The warm numbers are for reference only.
Performance
Host/port resolution is called three times per connect (validation, the folded Host, and the proxy endpoint). Each call now parses with
net/url. The table gives the median of 10go test -benchruns per case, run one at a time on the same machine, for 52bd5ac and this commit. Allocations are per call.validateKernelHostPortvalidateKernelHostPorthost:port, embeddedvalidateKernelHostPortkernelHostWithPortkernelHostWithPorthost:port, embeddedkernelHostWithPorthost:port, embeddedThe
net/urlparse costs roughly 170 to 200 ns per call. On the connect path that is about 0.5 to 0.9 µs in total, against a connect that takes milliseconds or more. The 52bd5achost:portfold was a 2 ns early return, and this commit parses even then to keep the rejection rules.Related Tickets & Documents
Related PRs: databricks/databricks-sql-python#1071, databricks/databricks-sql-nodejs#636, https://github.com/databricks/databricks-driver-test/pull/2188
This pull request and its description were written by Isaac.