Skip to content

fix(kernel): honor a custom port over HTTPS on the kernel backend - #636

Open
cathleeny wants to merge 6 commits into
mainfrom
cathleeny/fix/kernel-honor-https-port
Open

cathleeny wants to merge 6 commits into
mainfrom
cathleeny/fix/kernel-honor-https-port

Conversation

@cathleeny

@cathleeny cathleeny commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

What type of PR is this?

  • Refactor
  • Feature
  • Bug Fix
  • Other

Description

On the kernel backend (useKernel), port was folded into the host only when https: false was set. Over HTTPS, port: 8443 was dropped and the kernel connected to 443, while the Thrift backend honored the port. This PR folds a supplied non-default port into a bare host for HTTPS too.

Precedence, matching the kernel path on origin/main (buildKernelHostName in lib/kernel/KernelAuth.ts):

  • A port embedded in host always wins. On origin/main buildKernelHostName returned options.host unchanged for HTTPS, and for https: false it kept an embedded port (hasPort) and appended :port only when the host had none. So {host: 'h:8443', port: 443} connected to 8443 and {host: 'localhost:3015', port: 80, https: false} connected to 3015. Both still do.
  • The explicit port applies only to a bare host. When it differs from the embedded port (and is not 443), the driver logs a warning naming both ports and connects to the embedded port. It does not throw. 443 is not reported, because getConnectionOptions fills 443 in as the default, so an explicit 443 cannot be told apart from it.
  • A falsy port (unset, 0, NaN, '', null, false) means the default, as on Thrift (options.port || 443) and on origin/main, where these never threw. No :443 is added. An explicit port of 443 adds nothing.
  • A bare IPv6 host is bracketed before a port is appended ([::1]:8443, http://[::1]:3015). With no port the host is unchanged.
  • A decimal string port such as '8443' is accepted, as Thrift accepts it.
  • A trailing / is dropped from host before any port handling, as the kernel does. A host with a path, query or fragment (/, ?, # after that trim) throws HiveDriverError when the port would be folded into it (https: false, or a non-443 port over HTTPS). With no port, or with 443 over HTTPS, such a host passes through unchanged, as on origin/main, including a host with a port after the path (h.io/path:8443), which base passed to the kernel verbatim. On origin/main such a host was passed to the kernel verbatim, and the kernel appends its API paths after the host (for example https://h.io/path/sql/...), so it was not expected to reach the warehouse (not exercised live). The check runs before the embedded-port parse, so h.io/path:8443 is rejected rather than read as port 8443. Without a port the host is passed through unchanged, as on origin/main.
  • Truthy invalid ports (non-numeric strings, true, out of range) and out-of-range embedded ports still throw HiveDriverError. Thrift also fails on these.

Thrift is not changed. On origin/main Thrift builds scheme://host:port/path from options.port, which getConnectionOptions always sets (443 by default), so an embedded port becomes h:8443:443, which node-fetch rejects with Invalid URL. Thrift cannot use an embedded port today. That is pre-existing; this PR keeps the kernel's precedence and does not change Thrift. A Thrift test (tests/unit/DBSQLClient.test.ts) pins that current URL, so any future Thrift change shows up as a test diff.

Documented, not guarded: with a non-default port, OAuth on the kernel backend reaches its endpoints (OIDC discovery and the token exchange) at the same host:port as the connection. Thrift keeps OAuth on the bare host (port 443). Noted in CONNECTION_PARAMETERS.md (port row) and the CHANGELOG.

How is this tested?

  • Unit tests

  • E2E Tests

  • Manually

  • N/A

  • npm test: 1808 passing, 0 failing. npm run lint, npm run prettier, npm run type-check: clean.

  • tests/unit/kernel/connectionOptions.test.ts: table cases for accepted, falsy and rejected ports across HTTPS and https: false, the embedded-port precedence (including {host:'h:8443', port:443} and {host:'localhost:3015', port:80, https:false}), trailing-slash handling, rejected path/query/fragment hosts (including h.io/path:8443 and the DBSQLClient default 443), and the conflict warning (logger spy: warns when ports differ, does not warn when they agree or the explicit port is 443).

  • tests/unit/DBSQLClient.test.ts: Thrift falsy-port defaults (443 over HTTPS, 80 over HTTP), and the pinned Thrift URLs for a host with an embedded port (https://h:8443:443/, http://localhost:3015:80/), which are unchanged by this PR.

  • Loopback on the built dist/ with the kernel native binding: two listeners on 127.0.0.1, P1 (embedded) and P2 (explicit). host: 127.0.0.1:P1, port: P2 sends a TLS ClientHello (first byte 0x16) to P1 and nothing to P2. The same holds for port: NaN, port: 443 and a trailing slash (host: 127.0.0.1:P1/). host: 127.0.0.1, port: P1 sends the ClientHello to P1. Re-run on the final dist.

  • Live, kernel path (KernelBackend), via the approved wrapper (live_matrix.py), final head a4c91da, DBSQL 417ec0c098c5d0b6 and Reyden 0000000000e01f56. Each case: connect, SELECT 1, fetch, close, with the effective path confirmed as KernelBackend.

    • Bare host, port: 443: DBSQL pass (warm 700/887/665 ms); Reyden pass (446/440/437 ms).
    • Host carrying :443, no port: DBSQL pass (623/806/776 ms); Reyden pass (464/449/446 ms).
    • Host carrying :443, port: 8443: DBSQL pass (631/750/644 ms); Reyden pass (446/444/450 ms). Connects to 443; the warning is recorded.
  • Reyden warm-up: the first query on the warehouse cold-started past the wrapper's 130 s limit in earlier attempts (fetchAll deadline at 63 to 64 s). A warm-up case with a 100 s per-phase deadline succeeded (warm 487/463/460 ms), and the Reyden cases above ran against the warm warehouse. Earlier attempts are recorded in the working notes, not in this PR.

Related PRs: databricks/databricks-sql-python#1071, databricks/databricks-sql-go#581, https://github.com/databricks/databricks-driver-test/pull/2188

This pull request and its description were written by Isaac.

The kernel backend folded `port` into the host only when `https: false` was set,
so `port: 8443` over HTTPS connected to 443 while the Thrift backend honored it.

buildKernelHostName folds a supplied non-default port into the host for HTTPS too:

- A bare host with a non-default port becomes `host:port`. `port: 443`, an omitted
  port, and `0` (the Thrift `options.port || 443` default) leave the host unchanged.
- A bare IPv6 host gets brackets before the port is appended, on both HTTPS and
  `https: false` (`[::1]:8443`, `http://[::1]:3015`). With no port the host is unchanged.
- A decimal string port such as `'8443'` is accepted, as on Thrift. Non-numeric or
  out-of-range values throw HiveDriverError.
- A port already in `host` is kept, and an explicit `port` must match it. A different
  one throws HiveDriverError, so a port is never dropped silently.

Documented, not guarded: with a non-default port, OAuth on the kernel backend reaches
its endpoints at the same `host:port` (Thrift keeps OAuth on the bare host). This is
noted in CONNECTION_PARAMETERS.md and CHANGELOG.md.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

✅ No issues identified.

Comment thread lib/kernel/KernelAuth.ts Resolved
Comment thread lib/kernel/KernelAuth.ts Resolved
cathleeny and others added 5 commits November 19, 2026 23:06
…ault

The kernel path folds an explicit `port` into `host`. Review feedback showed
two regressions from the first version of this change, and this commit fixes
them to keep the pre-existing kernel precedence:

- A port embedded in `host` wins over `port`, as on origin/main, where
  `{host: 'h:8443', port: 443}` connected to 8443 and
  `{host: 'localhost:3015', port: 80, https: false}` connected to 3015.
  A differing explicit port is logged as a warning and not thrown. An explicit
  443 is treated as the default (getConnectionOptions fills it in) and is not
  reported.
- A falsy `port` (0, NaN, '', null, undefined, false) means the default, as on
  Thrift (`options.port || 443`). It no longer throws.
- A trailing "/" is dropped before port handling. A path, query or fragment is
  rejected only when a port has to be folded into the host, since that would
  produce a malformed authority. Inputs with no port are unchanged.
- Truthy invalid ports (non-numeric strings, out of range, true) still throw,
  as Thrift also fails on them.

The logger is passed from KernelBackend into buildKernelConnectionOptions.

Tests: table-driven cases in connectionOptions.test.ts cover the precedence,
falsy ports, the warning, and rejection of path and fragment hosts. A Thrift
case in DBSQLClient.test.ts covers falsy-port defaults.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
A host with a path, query or fragment cannot connect on the kernel path,
because the kernel puts its API paths after the host. Before this commit,
`host: 'h.io/path:8443'` also parsed as host `h.io/path` with embedded port
8443, which passed a broken host straight through. With an explicit `port`
(DBSQLClient always supplies 443 by default) such a host now throws
HiveDriverError before the embedded-port parse. Without a `port` the host is
passed through unchanged, as before.

Thrift is unchanged. A test in DBSQLClient.test.ts pins the current Thrift
URL for a host with an embedded port (`https://h:8443:443/`), which
node-fetch rejects, so the kernel fix does not alter Thrift.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
…e path

The path, query and fragment check from the previous commit rejected any
explicit port, including the default 443 that DBSQLClient always supplies.
That broke inputs that passed through on origin/main: `h/x` with no port or
with port 443 over HTTPS was passed to the kernel verbatim before this change.

The check now fires only when the port would be folded into the host
(`https: false`, or a non-443 port over HTTPS) or when the host carries a
port after the path (`h.io/path:8443`), which must not be read as port 8443.
With no port, or with 443 over HTTPS and no embedded port, the host passes
through unchanged. The check still runs before the embedded-port parse.

Tests cover the pass-through cases (no port, port 443, https://host/x) and
keep the rejection cases for the folded and embedded-after-path forms.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
The path, query and fragment check now fires only when the explicit port is
folded into the host (https: false, or a non-443 port over HTTPS). A host
with no folded port passes through unchanged, as on origin/main, including
a host with a port after the path (h.io/path:8443), which base passed to the
kernel verbatim. The check still runs before the embedded-port parse.

Tests keep the rejection cases for the folded form and add pass-through
cases for no port, port 443, and an embedded port after a path.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Resolve the CHANGELOG.md conflict by keeping both Unreleased entries: this PR's
kernel port entry and main's Arrow row materialization entry. No code conflicts.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

Verdict: 1 Low

Looks good, with 1 low concern. Folding the port into the host keeps the old precedence for embedded ports, and the tests cover the cases well. The one concern is the exemption for port 443 in the warning: its stated reason doesn't hold, because the kernel receives the raw caller options.

Comment thread lib/kernel/KernelAuth.ts
throw new HiveDriverError(`kernel backend: host carries port ${embedded}, which is outside 1..65535.`);
}
// 443 is the default that getConnectionOptions fills in, so it is not reported as a conflict.
if (port !== undefined && port !== 443 && embedded !== undefined && embedded !== port) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — The warning skips port 443 because the comment says getConnectionOptions fills it in. But backend.connect(options) gets the raw caller options, so a 443 was set explicitly. {host:'h:8443', port:443} therefore drops the caller's port without a warning. Remove the port !== 443 exemption, or correct the comment.

🔀 Found by Claude Opus 5.5 · confirmed by the judge

This branch was previously deployed

1 inactive deployment
azure-prod — 23ffcf69 Deployed Nov 20, 2026 by cathleeny via e2e-test (24.22.1, 24_22_1) #2170
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants