Repository navigation
Conversation
The kernel backend folded `port` into the host only when `https: false` was set, so `port: 8443` over HTTPS connected to 443 while the Thrift backend honored it. buildKernelHostName folds a supplied non-default port into the host for HTTPS too: - A bare host with a non-default port becomes `host:port`. `port: 443`, an omitted port, and `0` (the Thrift `options.port || 443` default) leave the host unchanged. - A bare IPv6 host gets brackets before the port is appended, on both HTTPS and `https: false` (`[::1]:8443`, `http://[::1]:3015`). With no port the host is unchanged. - A decimal string port such as `'8443'` is accepted, as on Thrift. Non-numeric or out-of-range values throw HiveDriverError. - A port already in `host` is kept, and an explicit `port` must match it. A different one throws HiveDriverError, so a port is never dropped silently. Documented, not guarded: with a non-default port, OAuth on the kernel backend reaches its endpoints at the same `host:port` (Thrift keeps OAuth on the bare host). This is noted in CONNECTION_PARAMETERS.md and CHANGELOG.md. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
…ault
The kernel path folds an explicit `port` into `host`. Review feedback showed
two regressions from the first version of this change, and this commit fixes
them to keep the pre-existing kernel precedence:
- A port embedded in `host` wins over `port`, as on origin/main, where
`{host: 'h:8443', port: 443}` connected to 8443 and
`{host: 'localhost:3015', port: 80, https: false}` connected to 3015.
A differing explicit port is logged as a warning and not thrown. An explicit
443 is treated as the default (getConnectionOptions fills it in) and is not
reported.
- A falsy `port` (0, NaN, '', null, undefined, false) means the default, as on
Thrift (`options.port || 443`). It no longer throws.
- A trailing "/" is dropped before port handling. A path, query or fragment is
rejected only when a port has to be folded into the host, since that would
produce a malformed authority. Inputs with no port are unchanged.
- Truthy invalid ports (non-numeric strings, out of range, true) still throw,
as Thrift also fails on them.
The logger is passed from KernelBackend into buildKernelConnectionOptions.
Tests: table-driven cases in connectionOptions.test.ts cover the precedence,
falsy ports, the warning, and rejection of path and fragment hosts. A Thrift
case in DBSQLClient.test.ts covers falsy-port defaults.
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
A host with a path, query or fragment cannot connect on the kernel path, because the kernel puts its API paths after the host. Before this commit, `host: 'h.io/path:8443'` also parsed as host `h.io/path` with embedded port 8443, which passed a broken host straight through. With an explicit `port` (DBSQLClient always supplies 443 by default) such a host now throws HiveDriverError before the embedded-port parse. Without a `port` the host is passed through unchanged, as before. Thrift is unchanged. A test in DBSQLClient.test.ts pins the current Thrift URL for a host with an embedded port (`https://h:8443:443/`), which node-fetch rejects, so the kernel fix does not alter Thrift. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
…e path The path, query and fragment check from the previous commit rejected any explicit port, including the default 443 that DBSQLClient always supplies. That broke inputs that passed through on origin/main: `h/x` with no port or with port 443 over HTTPS was passed to the kernel verbatim before this change. The check now fires only when the port would be folded into the host (`https: false`, or a non-443 port over HTTPS) or when the host carries a port after the path (`h.io/path:8443`), which must not be read as port 8443. With no port, or with 443 over HTTPS and no embedded port, the host passes through unchanged. The check still runs before the embedded-port parse. Tests cover the pass-through cases (no port, port 443, https://host/x) and keep the rejection cases for the folded and embedded-after-path forms. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
The path, query and fragment check now fires only when the explicit port is folded into the host (https: false, or a non-443 port over HTTPS). A host with no folded port passes through unchanged, as on origin/main, including a host with a port after the path (h.io/path:8443), which base passed to the kernel verbatim. The check still runs before the embedded-port parse. Tests keep the rejection cases for the folded form and add pass-through cases for no port, port 443, and an embedded port after a path. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Resolve the CHANGELOG.md conflict by keeping both Unreleased entries: this PR's kernel port entry and main's Arrow row materialization entry. No code conflicts. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Looks good, with 1 low concern. Folding the port into the host keeps the old precedence for embedded ports, and the tests cover the cases well. The one concern is the exemption for port 443 in the warning: its stated reason doesn't hold, because the kernel receives the raw caller options.
| throw new HiveDriverError(`kernel backend: host carries port ${embedded}, which is outside 1..65535.`); | ||
| } | ||
| // 443 is the default that getConnectionOptions fills in, so it is not reported as a conflict. | ||
| if (port !== undefined && port !== 443 && embedded !== undefined && embedded !== port) { |
There was a problem hiding this comment.
🔵 Low — The warning skips port 443 because the comment says getConnectionOptions fills it in. But backend.connect(options) gets the raw caller options, so a 443 was set explicitly. {host:'h:8443', port:443} therefore drops the caller's port without a warning. Remove the port !== 443 exemption, or correct the comment.
🔀 Found by Claude Opus 5.5 · confirmed by the judge
What type of PR is this?
Description
On the kernel backend (
useKernel),portwas folded into the host only whenhttps: falsewas set. Over HTTPS,port: 8443was dropped and the kernel connected to 443, while the Thrift backend honored the port. This PR folds a supplied non-default port into a bare host for HTTPS too.Precedence, matching the kernel path on origin/main (
buildKernelHostNameinlib/kernel/KernelAuth.ts):hostalways wins. On origin/mainbuildKernelHostNamereturnedoptions.hostunchanged for HTTPS, and forhttps: falseit kept an embedded port (hasPort) and appended:portonly when the host had none. So{host: 'h:8443', port: 443}connected to 8443 and{host: 'localhost:3015', port: 80, https: false}connected to 3015. Both still do.portapplies only to a bare host. When it differs from the embedded port (and is not 443), the driver logs a warning naming both ports and connects to the embedded port. It does not throw. 443 is not reported, becausegetConnectionOptionsfills 443 in as the default, so an explicit 443 cannot be told apart from it.port(unset,0,NaN,'',null,false) means the default, as on Thrift (options.port || 443) and on origin/main, where these never threw. No:443is added. An explicit port of 443 adds nothing.[::1]:8443,http://[::1]:3015). With no port the host is unchanged.'8443'is accepted, as Thrift accepts it./is dropped fromhostbefore any port handling, as the kernel does. A host with a path, query or fragment (/,?,#after that trim) throwsHiveDriverErrorwhen the port would be folded into it (https: false, or a non-443 port over HTTPS). With no port, or with 443 over HTTPS, such a host passes through unchanged, as on origin/main, including a host with a port after the path (h.io/path:8443), which base passed to the kernel verbatim. On origin/main such a host was passed to the kernel verbatim, and the kernel appends its API paths after the host (for examplehttps://h.io/path/sql/...), so it was not expected to reach the warehouse (not exercised live). The check runs before the embedded-port parse, soh.io/path:8443is rejected rather than read as port 8443. Without aportthe host is passed through unchanged, as on origin/main.true, out of range) and out-of-range embedded ports still throwHiveDriverError. Thrift also fails on these.Thrift is not changed. On origin/main Thrift builds
scheme://host:port/pathfromoptions.port, whichgetConnectionOptionsalways sets (443 by default), so an embedded port becomesh:8443:443, which node-fetch rejects withInvalid URL. Thrift cannot use an embedded port today. That is pre-existing; this PR keeps the kernel's precedence and does not change Thrift. A Thrift test (tests/unit/DBSQLClient.test.ts) pins that current URL, so any future Thrift change shows up as a test diff.Documented, not guarded: with a non-default
port, OAuth on the kernel backend reaches its endpoints (OIDC discovery and the token exchange) at the samehost:portas the connection. Thrift keeps OAuth on the bare host (port 443). Noted inCONNECTION_PARAMETERS.md(port row) and the CHANGELOG.How is this tested?
Unit tests
E2E Tests
Manually
N/A
npm test: 1808 passing, 0 failing.npm run lint,npm run prettier,npm run type-check: clean.tests/unit/kernel/connectionOptions.test.ts: table cases for accepted, falsy and rejected ports across HTTPS andhttps: false, the embedded-port precedence (including{host:'h:8443', port:443}and{host:'localhost:3015', port:80, https:false}), trailing-slash handling, rejected path/query/fragment hosts (includingh.io/path:8443and the DBSQLClient default 443), and the conflict warning (logger spy: warns when ports differ, does not warn when they agree or the explicit port is 443).tests/unit/DBSQLClient.test.ts: Thrift falsy-port defaults (443 over HTTPS, 80 over HTTP), and the pinned Thrift URLs for a host with an embedded port (https://h:8443:443/,http://localhost:3015:80/), which are unchanged by this PR.Loopback on the built
dist/with the kernel native binding: two listeners on127.0.0.1, P1 (embedded) and P2 (explicit).host: 127.0.0.1:P1, port: P2sends a TLS ClientHello (first byte0x16) to P1 and nothing to P2. The same holds forport: NaN,port: 443and a trailing slash (host: 127.0.0.1:P1/).host: 127.0.0.1, port: P1sends the ClientHello to P1. Re-run on the final dist.Live, kernel path (
KernelBackend), via the approved wrapper (live_matrix.py), final head a4c91da, DBSQL417ec0c098c5d0b6and Reyden0000000000e01f56. Each case: connect,SELECT 1, fetch, close, with the effective path confirmed asKernelBackend.port: 443: DBSQL pass (warm 700/887/665 ms); Reyden pass (446/440/437 ms).:443, no port: DBSQL pass (623/806/776 ms); Reyden pass (464/449/446 ms).:443,port: 8443: DBSQL pass (631/750/644 ms); Reyden pass (446/444/450 ms). Connects to 443; the warning is recorded.Reyden warm-up: the first query on the warehouse cold-started past the wrapper's 130 s limit in earlier attempts (fetchAll deadline at 63 to 64 s). A warm-up case with a 100 s per-phase deadline succeeded (warm 487/463/460 ms), and the Reyden cases above ran against the warm warehouse. Earlier attempts are recorded in the working notes, not in this PR.
Related PRs: databricks/databricks-sql-python#1071, databricks/databricks-sql-go#581, https://github.com/databricks/databricks-driver-test/pull/2188
This pull request and its description were written by Isaac.