From d2e2169f4f877d4628f22b9d00247466df2479c9 Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 17:25:16 +0000 Subject: [PATCH 1/5] fix(kernel): honor a custom port over HTTPS on the kernel backend The kernel backend folded `port` into the host only when `https: false` was set, so `port: 8443` over HTTPS connected to 443 while the Thrift backend honored it. buildKernelHostName folds a supplied non-default port into the host for HTTPS too: - A bare host with a non-default port becomes `host:port`. `port: 443`, an omitted port, and `0` (the Thrift `options.port || 443` default) leave the host unchanged. - A bare IPv6 host gets brackets before the port is appended, on both HTTPS and `https: false` (`[::1]:8443`, `http://[::1]:3015`). With no port the host is unchanged. - A decimal string port such as `'8443'` is accepted, as on Thrift. Non-numeric or out-of-range values throw HiveDriverError. - A port already in `host` is kept, and an explicit `port` must match it. A different one throws HiveDriverError, so a port is never dropped silently. Documented, not guarded: with a non-default port, OAuth on the kernel backend reaches its endpoints at the same `host:port` (Thrift keeps OAuth on the bare host). This is noted in CONNECTION_PARAMETERS.md and CHANGELOG.md. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 1 + CONNECTION_PARAMETERS.md | 18 +++--- lib/kernel/KernelAuth.ts | 58 ++++++++++++++---- tests/unit/kernel/connectionOptions.test.ts | 66 ++++++++++++++++++++- 4 files changed, 121 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 938983a3..cc4fed04 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## Unreleased +- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, `0` and an omitted port leave `host` unchanged; a numeric string is accepted; a port that conflicts with one in `host`, or is out of range, throws `HiveDriverError`. With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). - The Thrift token federation exchange (`enableTokenFederation`) now goes through the connection's agent, so `proxy`, `checkServerCertificate`, `customCaCert` and `clientCert`/`clientKey` apply to it. Previously it connected directly with Node's default agent, so behind a required proxy the exchange failed and the driver fell back to the original token. (#611) - Kernel backend: a failed status poll while waiting for a statement (a `runAsync: true` statement, or a default execution that outlives the server's inline wait) and a failed result fetch in `hasMoreRows()` now throw the mapped driver error, such as `HiveDriverError: HTTP 500: Internal Server Error` once retries on HTTP 5xx run out, instead of a raw `Error` whose message is the internal `__databricks_error__:{...}` envelope. As in `fetchChunk()`, a failed fetch in `hasMoreRows()` also closes the statement. (#608) - Honor `cloudFetchConcurrentDownloads` when opening kernel sessions, with a default window of 16 and explicit chunk-setting overrides. Values other than positive safe integers raise `HiveDriverError` at session open. diff --git a/CONNECTION_PARAMETERS.md b/CONNECTION_PARAMETERS.md index edba6002..c2b4a7f6 100644 --- a/CONNECTION_PARAMETERS.md +++ b/CONNECTION_PARAMETERS.md @@ -38,15 +38,15 @@ backend (the default), the **SEA / Kernel** backend, or both. ## Connection identity and backend selection -| Option | Type | Thrift | Kernel | Default Value | Note | -| ----------------------- | --------- | :----: | :----: | ------------- | -------------------------------------------------------------------------------------- | -| `host` | `string` | ✅ | ✅ | — (required) | Required on both. | -| `path` | `string` | ✅ | ✅ | — (required) | Kernel also derives `x-databricks-org-id` from a `?o=` query parameter. | -| `port` | `number` | ✅ | ⚠️ | `443` / — | Kernel uses it only with `https: false`; otherwise put a non-default port in `host`. | -| `https` | `boolean` | ✅ | ✅ | `true` | `false` selects plain HTTP; see [Plain HTTP transport](#plain-http-transport). | -| `userAgentEntry` | `string` | ✅ | ✅ | — | Folded into the connector `User-Agent` on both. | -| `useKernel` (internal) | `boolean` | — | — | `false` | Selects the backend: `false`/omitted → Thrift, `true` → Kernel. | -| `clientId` (deprecated) | `string` | ✅ | ✅ | — | Runtime-only alias for `userAgentEntry`. `userAgentEntry` wins when both are supplied. | +| Option | Type | Thrift | Kernel | Default Value | Note | +| ----------------------- | --------- | :----: | :----: | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `host` | `string` | ✅ | ✅ | — (required) | Required on both. | +| `path` | `string` | ✅ | ✅ | — (required) | Kernel also derives `x-databricks-org-id` from a `?o=` query parameter. | +| `port` | `number` | ✅ | ✅ | `443` / — | Kernel folds a non-default port into `host` over HTTPS and with `https: false` (a bare IPv6 host is bracketed); a port also in `host` must match. With OAuth, the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare `host` (port 443). | +| `https` | `boolean` | ✅ | ✅ | `true` | `false` selects plain HTTP; see [Plain HTTP transport](#plain-http-transport). | +| `userAgentEntry` | `string` | ✅ | ✅ | — | Folded into the connector `User-Agent` on both. | +| `useKernel` (internal) | `boolean` | — | — | `false` | Selects the backend: `false`/omitted → Thrift, `true` → Kernel. | +| `clientId` (deprecated) | `string` | ✅ | ✅ | — | Runtime-only alias for `userAgentEntry`. `userAgentEntry` wins when both are supplied. | Set `client.getConfig().useCloudFetch = false` (default `true`) before opening a session to fetch complete results without cloud-storage downloads. On the kernel backend this diff --git a/lib/kernel/KernelAuth.ts b/lib/kernel/KernelAuth.ts index 8d900485..cc4a86d2 100644 --- a/lib/kernel/KernelAuth.ts +++ b/lib/kernel/KernelAuth.ts @@ -830,20 +830,54 @@ export function buildKernelProxyOptions(options: ConnectionOptions): KernelProxy return { proxy: out }; } -// The kernel prepends `https://` to a bare host but keeps an explicit scheme. -// It otherwise ignores `port`, so carry it into the URL unless `host` already has one. +// `name:port` or `[ipv6]:port`; a bare IPv6 literal like `::1` has no port. +const EMBEDDED_PORT_RE = /^(\[[^\]]+\]|[^:]+):(\d+)$/; + +// An unset, null or zero port means the default, as on Thrift (`options.port || 443`). +// A decimal string such as `'8443'` is accepted, as Thrift accepts it. +function validateKernelPort(port: unknown): number | undefined { + if (port === undefined || port === null) return undefined; + let value = Number.NaN; + if (typeof port === 'number') value = port; + else if (typeof port === 'string' && /^[0-9]+$/.test(port)) value = Number(port); + if (value === 0) return undefined; + if (!Number.isInteger(value) || value < 1 || value > 65535) { + throw new HiveDriverError(`kernel backend: \`port\` must be an integer between 1 and 65535; got ${String(port)}.`); + } + return value; +} + +// A bare IPv6 literal (colons, no brackets) needs brackets before a port is appended. +const bracketIpv6 = (authority: string): string => + authority.includes(':') && !authority.startsWith('[') ? `[${authority}]` : authority; + +// The kernel prepends `https://` to a bare host and keeps an explicit scheme, but it +// ignores the separate `port`, so a supplied port is folded into the host here. +// HTTPS: a bare host is unchanged unless the port is non-default (not 443). +// `https: false`: the scheme is http:// and a supplied port is always folded in. +// A port in both `host` and `port` must agree. function buildKernelHostName(options: ConnectionOptions): string { - if (options.https !== false) { - return options.host; + const port = validateKernelPort(options.port); + const scheme = /^https?:\/\//i.exec(options.host)?.[0] ?? ''; + const authority = options.host.slice(scheme.length).replace(/\/$/, ''); + const match = EMBEDDED_PORT_RE.exec(authority); + const embedded = match ? Number(match[2]) : undefined; + if (embedded !== undefined && (embedded < 1 || embedded > 65535)) { + throw new HiveDriverError(`kernel backend: host carries port ${embedded}, which is outside 1..65535.`); + } + if (port !== undefined && embedded !== undefined && embedded !== port) { + throw new HiveDriverError( + `kernel backend: host ${JSON.stringify(options.host)} carries port ${embedded} but \`port\` is ${port}. ` + + 'Set the port in one place only.', + ); + } + if (options.https === false) { + return port !== undefined && embedded === undefined + ? `http://${bracketIpv6(authority)}:${port}` + : `http://${authority}`; } - // The kernel accepts a full URL in `host`, so drop an existing `http://` or - // `https://` scheme before applying HTTP (`https: false` selects the scheme). - // Then strip a trailing `/` like the Thrift URL builder does. - const host = options.host.replace(/^https?:\/\//i, '').replace(/\/$/, ''); - // `name:port` or `[ipv6]:port`; a bare IPv6 literal like `::1` has no port. - const hasPort = /^(\[[^\]]+\]|[^:]+):\d+$/.test(host); - const port = options.port && !hasPort ? `:${options.port}` : ''; - return `http://${host}${port}`; + if (port === undefined || port === 443 || embedded !== undefined) return options.host; + return `${scheme}${bracketIpv6(authority)}:${port}`; } export function buildKernelConnectionOptions(options: ConnectionOptions): KernelNativeConnectionOptions { diff --git a/tests/unit/kernel/connectionOptions.test.ts b/tests/unit/kernel/connectionOptions.test.ts index 153e388f..6d51a88c 100644 --- a/tests/unit/kernel/connectionOptions.test.ts +++ b/tests/unit/kernel/connectionOptions.test.ts @@ -127,11 +127,75 @@ describe('KernelAuth connection options — https transport', () => { ); // Same URL shape as the Thrift transport, which also requires brackets. expect(buildKernelConnectionOptions(opts({ host: '::1', port: 3015, https: false })).hostName).to.equal( - 'http://::1:3015', + 'http://[::1]:3015', ); }); }); +describe('KernelAuth connection options — port folding', () => { + const hostName = (extra: Record) => buildKernelConnectionOptions(opts(extra)).hostName; + + // [connection options, expected hostName] for HTTPS (default) and https: false. + const accepted: Array<[Record, string]> = [ + [{ port: 8443 }, 'h.databricks.com:8443'], + [{ port: 8443, https: true }, 'h.databricks.com:8443'], + [{ port: '8443' }, 'h.databricks.com:8443'], // a decimal string is accepted, as on Thrift + [{ port: 443 }, 'h.databricks.com'], // no :443 suffix + [{ port: 0 }, 'h.databricks.com'], // zero means the default, as on Thrift + [{}, 'h.databricks.com'], + [{ host: 'h.databricks.com:8443' }, 'h.databricks.com:8443'], + [{ host: 'h.databricks.com:8443', port: 8443 }, 'h.databricks.com:8443'], + [{ host: 'https://h.databricks.com', port: 8443 }, 'https://h.databricks.com:8443'], + [{ host: '[::1]', port: 8443 }, '[::1]:8443'], + [{ host: '[::1]:8443', port: 8443 }, '[::1]:8443'], + [{ host: '::1', port: 8443 }, '[::1]:8443'], // bare IPv6 is bracketed + [{ host: '::1', port: 443 }, '::1'], + [{ host: 'h.databricks.com', port: 3015, https: false }, 'http://h.databricks.com:3015'], + [{ host: 'localhost', port: 80, https: false }, 'http://localhost:80'], + [{ host: 'localhost', port: '3015', https: false }, 'http://localhost:3015'], + [{ host: '::1', port: 3015, https: false }, 'http://[::1]:3015'], // bare IPv6 is bracketed + [{ host: '::1', https: false }, 'http://::1'], // no port: unchanged + [{ host: 'http://localhost:3015', https: false }, 'http://localhost:3015'], + ]; + it('folds the port into the host as the table specifies', () => { + for (const [extra, want] of accepted) { + expect(hostName(extra), JSON.stringify(extra)).to.equal(want); + } + }); + + // [connection options, message] that must throw HiveDriverError. + const badPorts: unknown[] = [ + 65536, + -1, + 1.5, + Number.NaN, + Number.POSITIVE_INFINITY, + true, + 'abc', + '8443x', + '', + '-1', + '65536', + ]; + const rejected: Array<[Record, RegExp]> = [ + ...badPorts.flatMap( + (port): Array<[Record, RegExp]> => [ + [{ port }, /must be an integer between 1 and 65535/], + [{ port, https: false }, /must be an integer between 1 and 65535/], + ], + ), + [{ host: 'h.databricks.com:7000', port: 8443 }, /carries port 7000 but `port` is 8443/], + [{ host: 'h.databricks.com:8443', port: 443 }, /carries port 8443 but `port` is 443/], + [{ host: '[::1]:7000', port: 8443 }, /carries port 7000/], + [{ host: 'h.databricks.com:70000' }, /outside 1\.\.65535/], + ]; + it('rejects invalid ports and port conflicts', () => { + for (const [extra, message] of rejected) { + expect(() => hostName(extra), JSON.stringify(extra)).to.throw(HiveDriverError, message); + } + }); +}); + describe('KernelAuth connection options — maxConnections', () => { it('forwards a valid positive integer', () => { const native = buildKernelConnectionOptions(opts({ maxConnections: 10 })) as { maxConnections?: number }; From 97d67cbddd30198668d380ba53659bfa71f6dfaf Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 23:06:51 +0000 Subject: [PATCH 2/5] Kernel: embedded host port wins over explicit port; falsy port is default The kernel path folds an explicit `port` into `host`. Review feedback showed two regressions from the first version of this change, and this commit fixes them to keep the pre-existing kernel precedence: - A port embedded in `host` wins over `port`, as on origin/main, where `{host: 'h:8443', port: 443}` connected to 8443 and `{host: 'localhost:3015', port: 80, https: false}` connected to 3015. A differing explicit port is logged as a warning and not thrown. An explicit 443 is treated as the default (getConnectionOptions fills it in) and is not reported. - A falsy `port` (0, NaN, '', null, undefined, false) means the default, as on Thrift (`options.port || 443`). It no longer throws. - A trailing "/" is dropped before port handling. A path, query or fragment is rejected only when a port has to be folded into the host, since that would produce a malformed authority. Inputs with no port are unchanged. - Truthy invalid ports (non-numeric strings, out of range, true) still throw, as Thrift also fails on them. The logger is passed from KernelBackend into buildKernelConnectionOptions. Tests: table-driven cases in connectionOptions.test.ts cover the precedence, falsy ports, the warning, and rejection of path and fragment hosts. A Thrift case in DBSQLClient.test.ts covers falsy-port defaults. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 +- CONNECTION_PARAMETERS.md | 18 ++--- lib/kernel/KernelAuth.ts | 58 ++++++++++------ lib/kernel/KernelBackend.ts | 2 +- tests/unit/DBSQLClient.test.ts | 13 ++++ tests/unit/kernel/connectionOptions.test.ts | 77 ++++++++++++++++----- 6 files changed, 118 insertions(+), 52 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cc4fed04..623a1bd3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased -- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, `0` and an omitted port leave `host` unchanged; a numeric string is accepted; a port that conflicts with one in `host`, or is out of range, throws `HiveDriverError`. With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). +- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, a falsy `port` (`0`, `NaN`, `''`, `null`, `false`) and an omitted port leave `host` unchanged; a numeric string is accepted; a port embedded in `host` wins over a different `port` (a warning is logged); an out-of-range port throws `HiveDriverError`. With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). - The Thrift token federation exchange (`enableTokenFederation`) now goes through the connection's agent, so `proxy`, `checkServerCertificate`, `customCaCert` and `clientCert`/`clientKey` apply to it. Previously it connected directly with Node's default agent, so behind a required proxy the exchange failed and the driver fell back to the original token. (#611) - Kernel backend: a failed status poll while waiting for a statement (a `runAsync: true` statement, or a default execution that outlives the server's inline wait) and a failed result fetch in `hasMoreRows()` now throw the mapped driver error, such as `HiveDriverError: HTTP 500: Internal Server Error` once retries on HTTP 5xx run out, instead of a raw `Error` whose message is the internal `__databricks_error__:{...}` envelope. As in `fetchChunk()`, a failed fetch in `hasMoreRows()` also closes the statement. (#608) - Honor `cloudFetchConcurrentDownloads` when opening kernel sessions, with a default window of 16 and explicit chunk-setting overrides. Values other than positive safe integers raise `HiveDriverError` at session open. diff --git a/CONNECTION_PARAMETERS.md b/CONNECTION_PARAMETERS.md index c2b4a7f6..9bad42c3 100644 --- a/CONNECTION_PARAMETERS.md +++ b/CONNECTION_PARAMETERS.md @@ -38,15 +38,15 @@ backend (the default), the **SEA / Kernel** backend, or both. ## Connection identity and backend selection -| Option | Type | Thrift | Kernel | Default Value | Note | -| ----------------------- | --------- | :----: | :----: | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `host` | `string` | ✅ | ✅ | — (required) | Required on both. | -| `path` | `string` | ✅ | ✅ | — (required) | Kernel also derives `x-databricks-org-id` from a `?o=` query parameter. | -| `port` | `number` | ✅ | ✅ | `443` / — | Kernel folds a non-default port into `host` over HTTPS and with `https: false` (a bare IPv6 host is bracketed); a port also in `host` must match. With OAuth, the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare `host` (port 443). | -| `https` | `boolean` | ✅ | ✅ | `true` | `false` selects plain HTTP; see [Plain HTTP transport](#plain-http-transport). | -| `userAgentEntry` | `string` | ✅ | ✅ | — | Folded into the connector `User-Agent` on both. | -| `useKernel` (internal) | `boolean` | — | — | `false` | Selects the backend: `false`/omitted → Thrift, `true` → Kernel. | -| `clientId` (deprecated) | `string` | ✅ | ✅ | — | Runtime-only alias for `userAgentEntry`. `userAgentEntry` wins when both are supplied. | +| Option | Type | Thrift | Kernel | Default Value | Note | +| ----------------------- | --------- | :----: | :----: | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `host` | `string` | ✅ | ✅ | — (required) | Required on both. | +| `path` | `string` | ✅ | ✅ | — (required) | Kernel also derives `x-databricks-org-id` from a `?o=` query parameter. | +| `port` | `number` | ✅ | ✅ | `443` / — | Kernel folds a non-default port into a bare `host` over HTTPS and with `https: false` (a bare IPv6 host is bracketed). A port already in `host` wins over `port`, with a warning when they differ; a falsy `port` means the default. With OAuth, the kernel uses the same `host:port` for its OAuth endpoints; Thrift keeps OAuth on the bare `host` (port 443). | +| `https` | `boolean` | ✅ | ✅ | `true` | `false` selects plain HTTP; see [Plain HTTP transport](#plain-http-transport). | +| `userAgentEntry` | `string` | ✅ | ✅ | — | Folded into the connector `User-Agent` on both. | +| `useKernel` (internal) | `boolean` | — | — | `false` | Selects the backend: `false`/omitted → Thrift, `true` → Kernel. | +| `clientId` (deprecated) | `string` | ✅ | ✅ | — | Runtime-only alias for `userAgentEntry`. `userAgentEntry` wins when both are supplied. | Set `client.getConfig().useCloudFetch = false` (default `true`) before opening a session to fetch complete results without cloud-storage downloads. On the kernel backend this diff --git a/lib/kernel/KernelAuth.ts b/lib/kernel/KernelAuth.ts index cc4a86d2..4c19353c 100644 --- a/lib/kernel/KernelAuth.ts +++ b/lib/kernel/KernelAuth.ts @@ -833,10 +833,12 @@ export function buildKernelProxyOptions(options: ConnectionOptions): KernelProxy // `name:port` or `[ipv6]:port`; a bare IPv6 literal like `::1` has no port. const EMBEDDED_PORT_RE = /^(\[[^\]]+\]|[^:]+):(\d+)$/; -// An unset, null or zero port means the default, as on Thrift (`options.port || 443`). -// A decimal string such as `'8443'` is accepted, as Thrift accepts it. -function validateKernelPort(port: unknown): number | undefined { - if (port === undefined || port === null) return undefined; +// An explicit `port` is read as Thrift reads it (`options.port || 443`): a falsy value +// (unset, null, '', 0, NaN, false) means the default. A truthy value must be a whole +// port number, and a decimal string such as `'8443'` is accepted. Anything else throws, +// as Thrift also fails on it. +function explicitPort(port: unknown): number | undefined { + if (!port) return undefined; let value = Number.NaN; if (typeof port === 'number') value = port; else if (typeof port === 'string' && /^[0-9]+$/.test(port)) value = Number(port); @@ -851,13 +853,13 @@ function validateKernelPort(port: unknown): number | undefined { const bracketIpv6 = (authority: string): string => authority.includes(':') && !authority.startsWith('[') ? `[${authority}]` : authority; -// The kernel prepends `https://` to a bare host and keeps an explicit scheme, but it -// ignores the separate `port`, so a supplied port is folded into the host here. -// HTTPS: a bare host is unchanged unless the port is non-default (not 443). -// `https: false`: the scheme is http:// and a supplied port is always folded in. -// A port in both `host` and `port` must agree. -function buildKernelHostName(options: ConnectionOptions): string { - const port = validateKernelPort(options.port); +// The kernel keeps a `host:port` authority but has no port setter, so an explicit port is +// folded into the host. Precedence, as on the pre-existing kernel path: a port embedded in +// `host` always wins. The explicit `port` applies only to a bare host, and when it differs +// from the embedded port the driver warns and connects to the embedded port. A trailing +// "/" is dropped before any port handling, as the kernel does. +function buildKernelHostName(options: ConnectionOptions, logger?: IDBSQLLogger): string { + const port = explicitPort(options.port); const scheme = /^https?:\/\//i.exec(options.host)?.[0] ?? ''; const authority = options.host.slice(scheme.length).replace(/\/$/, ''); const match = EMBEDDED_PORT_RE.exec(authority); @@ -865,22 +867,34 @@ function buildKernelHostName(options: ConnectionOptions): string { if (embedded !== undefined && (embedded < 1 || embedded > 65535)) { throw new HiveDriverError(`kernel backend: host carries port ${embedded}, which is outside 1..65535.`); } - if (port !== undefined && embedded !== undefined && embedded !== port) { - throw new HiveDriverError( + // 443 is the default that getConnectionOptions fills in, so it is not reported as a conflict. + if (port !== undefined && port !== 443 && embedded !== undefined && embedded !== port) { + logger?.log( + LogLevel.warn, `kernel backend: host ${JSON.stringify(options.host)} carries port ${embedded} but \`port\` is ${port}. ` + - 'Set the port in one place only.', + `Connecting to port ${embedded}; \`port\` is ignored.`, ); } - if (options.https === false) { - return port !== undefined && embedded === undefined - ? `http://${bracketIpv6(authority)}:${port}` - : `http://${authority}`; + // Fold a port into a bare host. `https: false` always folds a supplied port, while HTTPS + // folds only a non-default one (443 is the default and adds nothing). + if (embedded === undefined && port !== undefined && (options.https === false || port !== 443)) { + if (/[/?#]/.test(authority)) { + throw new HiveDriverError( + `kernel backend: host ${JSON.stringify(options.host)} has a path, query or fragment, so \`port\` ${port} ` + + 'cannot be added to it. Use a bare host.', + ); + } + const hostPort = `${bracketIpv6(authority)}:${port}`; + return options.https === false ? `http://${hostPort}` : `${scheme}${hostPort}`; } - if (port === undefined || port === 443 || embedded !== undefined) return options.host; - return `${scheme}${bracketIpv6(authority)}:${port}`; + if (options.https === false) return `http://${authority}`; + return `${scheme}${authority}`; } -export function buildKernelConnectionOptions(options: ConnectionOptions): KernelNativeConnectionOptions { +export function buildKernelConnectionOptions( + options: ConnectionOptions, + logger?: IDBSQLLogger, +): KernelNativeConnectionOptions { const { authType } = options as { authType?: string }; const base: { @@ -893,7 +907,7 @@ export function buildKernelConnectionOptions(options: ConnectionOptions): Kernel KernelHttpOptions & KernelProxyOptions & KernelFederationOptions = { - hostName: buildKernelHostName(options), + hostName: buildKernelHostName(options, logger), httpPath: prependSlash(options.path), // Match the NodeJS Thrift driver, which surfaces INTERVAL columns as // strings. The kernel defaults to native Arrow interval/duration types; diff --git a/lib/kernel/KernelBackend.ts b/lib/kernel/KernelBackend.ts index 5c622ea0..e2357acc 100644 --- a/lib/kernel/KernelBackend.ts +++ b/lib/kernel/KernelBackend.ts @@ -96,7 +96,7 @@ export default class KernelBackend implements IBackend { // config (the same `ClientConfig` knobs the Thrift `HttpRetryPolicy` reads), // converted from the connector's milliseconds to the kernel's whole seconds. this.nativeOptions = { - ...buildKernelConnectionOptions(options), + ...buildKernelConnectionOptions(options, this.context.getLogger()), ...buildKernelRetryOptions(this.context.getConfig()), ...buildKernelTelemetryOptions(this.context.getConfig(), options, this.context.getLogger()), }; diff --git a/tests/unit/DBSQLClient.test.ts b/tests/unit/DBSQLClient.test.ts index 497889db..3638847a 100644 --- a/tests/unit/DBSQLClient.test.ts +++ b/tests/unit/DBSQLClient.test.ts @@ -234,6 +234,19 @@ describe('DBSQLClient.connect', () => { expect(connectionOptions.port).to.equal(3015); }); + it('treats a falsy port as the default (443 over HTTPS, 80 over HTTP), as the kernel path does', async () => { + const client = new DBSQLClient(); + + for (const port of [0, Number.NaN, undefined, null, '', false]) { + const label = String(port); + expect(client['getConnectionOptions']({ ...connectOptions, port: port as any }).port, label).to.equal(443); + expect( + client['getConnectionOptions']({ ...connectOptions, port: port as any, https: false }).port, + label, + ).to.equal(80); + } + }); + it('should only switch to plain HTTP for an explicit https: false', async () => { const client = new DBSQLClient(); diff --git a/tests/unit/kernel/connectionOptions.test.ts b/tests/unit/kernel/connectionOptions.test.ts index 6d51a88c..253e828b 100644 --- a/tests/unit/kernel/connectionOptions.test.ts +++ b/tests/unit/kernel/connectionOptions.test.ts @@ -21,6 +21,7 @@ import { } from '../../../lib/kernel/KernelAuth'; import { ConnectionOptions } from '../../../lib/contracts/IDBSQLClient'; import HiveDriverError from '../../../lib/errors/HiveDriverError'; +import IDBSQLLogger, { LogLevel } from '../../../lib/contracts/IDBSQLLogger'; const PAT = { host: 'h.databricks.com', path: '/sql/1.0/warehouses/abc', token: 'dapi-x' }; @@ -136,26 +137,50 @@ describe('KernelAuth connection options — port folding', () => { const hostName = (extra: Record) => buildKernelConnectionOptions(opts(extra)).hostName; // [connection options, expected hostName] for HTTPS (default) and https: false. + // An embedded host port always wins; the explicit port applies only to a bare host. const accepted: Array<[Record, string]> = [ [{ port: 8443 }, 'h.databricks.com:8443'], [{ port: 8443, https: true }, 'h.databricks.com:8443'], [{ port: '8443' }, 'h.databricks.com:8443'], // a decimal string is accepted, as on Thrift [{ port: 443 }, 'h.databricks.com'], // no :443 suffix - [{ port: 0 }, 'h.databricks.com'], // zero means the default, as on Thrift + // Falsy means the default, as on Thrift (`options.port || 443`); never a throw. + [{ port: 0 }, 'h.databricks.com'], + [{ port: '' }, 'h.databricks.com'], + [{ port: null }, 'h.databricks.com'], + [{ port: undefined }, 'h.databricks.com'], + [{ port: Number.NaN }, 'h.databricks.com'], + [{ port: false }, 'h.databricks.com'], + [{ port: Number.NaN, https: false }, 'http://h.databricks.com'], [{}, 'h.databricks.com'], [{ host: 'h.databricks.com:8443' }, 'h.databricks.com:8443'], [{ host: 'h.databricks.com:8443', port: 8443 }, 'h.databricks.com:8443'], + // A port embedded in host wins over a different explicit port (warned, not thrown). + [{ host: 'h.databricks.com:8443', port: 443 }, 'h.databricks.com:8443'], + [{ host: 'h.databricks.com:7000', port: 8443 }, 'h.databricks.com:7000'], + [{ host: '[::1]:7000', port: 8443 }, '[::1]:7000'], [{ host: 'https://h.databricks.com', port: 8443 }, 'https://h.databricks.com:8443'], [{ host: '[::1]', port: 8443 }, '[::1]:8443'], [{ host: '[::1]:8443', port: 8443 }, '[::1]:8443'], [{ host: '::1', port: 8443 }, '[::1]:8443'], // bare IPv6 is bracketed [{ host: '::1', port: 443 }, '::1'], + // Trailing "/" is dropped before port handling; a host with no port is otherwise unchanged. + [{ host: 'h.databricks.com/', port: 8443 }, 'h.databricks.com:8443'], + [{ host: 'https://h.databricks.com/', port: 8443 }, 'https://h.databricks.com:8443'], + [{ host: 'h.databricks.com:8443/' }, 'h.databricks.com:8443'], + [{ host: 'h.databricks.com/' }, 'h.databricks.com'], + [{ host: 'h.databricks.com/x' }, 'h.databricks.com/x'], // no port: unchanged, as before [{ host: 'h.databricks.com', port: 3015, https: false }, 'http://h.databricks.com:3015'], [{ host: 'localhost', port: 80, https: false }, 'http://localhost:80'], [{ host: 'localhost', port: '3015', https: false }, 'http://localhost:3015'], [{ host: '::1', port: 3015, https: false }, 'http://[::1]:3015'], // bare IPv6 is bracketed [{ host: '::1', https: false }, 'http://::1'], // no port: unchanged [{ host: 'http://localhost:3015', https: false }, 'http://localhost:3015'], + // Reviewer cases: the embedded port wins over the explicit one, on both transports' shape. + [{ host: 'h.databricks.com:8443', port: 443, https: true }, 'h.databricks.com:8443'], + [{ host: 'localhost:3015', port: 80, https: false }, 'http://localhost:3015'], + [{ host: 'localhost:3015', port: 443, https: false }, 'http://localhost:3015'], + [{ host: 'h.databricks.com:8443/', port: 443, https: false }, 'http://h.databricks.com:8443'], + [{ host: 'h.databricks.com/', port: 8443, https: false }, 'http://h.databricks.com:8443'], ]; it('folds the port into the host as the table specifies', () => { for (const [extra, want] of accepted) { @@ -163,20 +188,10 @@ describe('KernelAuth connection options — port folding', () => { } }); - // [connection options, message] that must throw HiveDriverError. - const badPorts: unknown[] = [ - 65536, - -1, - 1.5, - Number.NaN, - Number.POSITIVE_INFINITY, - true, - 'abc', - '8443x', - '', - '-1', - '65536', - ]; + // [connection options, message] that must throw HiveDriverError: truthy invalid ports + // (Thrift also fails on these), a path/query/fragment that would take a folded port, and + // out-of-range embedded ports. + const badPorts: unknown[] = [65536, -1, 1.5, Number.POSITIVE_INFINITY, true, 'abc', '8443x', '-1', '65536']; const rejected: Array<[Record, RegExp]> = [ ...badPorts.flatMap( (port): Array<[Record, RegExp]> => [ @@ -184,16 +199,40 @@ describe('KernelAuth connection options — port folding', () => { [{ port, https: false }, /must be an integer between 1 and 65535/], ], ), - [{ host: 'h.databricks.com:7000', port: 8443 }, /carries port 7000 but `port` is 8443/], - [{ host: 'h.databricks.com:8443', port: 443 }, /carries port 8443 but `port` is 443/], - [{ host: '[::1]:7000', port: 8443 }, /carries port 7000/], + [{ host: 'h.databricks.com/x', port: 8443 }, /has a path, query or fragment/], + [{ host: 'h.databricks.com?x', port: 8443 }, /has a path, query or fragment/], + [{ host: 'h.databricks.com#x', port: 3015, https: false }, /has a path, query or fragment/], [{ host: 'h.databricks.com:70000' }, /outside 1\.\.65535/], ]; - it('rejects invalid ports and port conflicts', () => { + it('rejects invalid ports and unfoldable hosts', () => { for (const [extra, message] of rejected) { expect(() => hostName(extra), JSON.stringify(extra)).to.throw(HiveDriverError, message); } }); + + it('warns and connects to the embedded port when an explicit port differs', () => { + const logs: Array<[LogLevel, string]> = []; + const logger = { + log: (level: LogLevel, message: string) => logs.push([level, message]), + } as unknown as IDBSQLLogger; + const native = buildKernelConnectionOptions(opts({ host: 'h.databricks.com:7000', port: 8443 }), logger); + expect(native.hostName).to.equal('h.databricks.com:7000'); + expect(logs).to.have.length(1); + expect(logs[0][0]).to.equal(LogLevel.warn); + expect(logs[0][1]).to.contain('carries port 7000'); + expect(logs[0][1]).to.contain('`port` is 8443'); + }); + + it('does not warn when the ports agree, the explicit port is the default, or there is no logger', () => { + const logs: Array<[LogLevel, string]> = []; + const logger = { + log: (level: LogLevel, message: string) => logs.push([level, message]), + } as unknown as IDBSQLLogger; + buildKernelConnectionOptions(opts({ host: 'h.databricks.com:8443', port: 8443 }), logger); + buildKernelConnectionOptions(opts({ host: 'h.databricks.com:8443', port: 443 }), logger); + buildKernelConnectionOptions(opts({ host: 'h.databricks.com:7000', port: 8443 })); + expect(logs).to.have.length(0); + }); }); describe('KernelAuth connection options — maxConnections', () => { From a4c91da9fd699fb541b09eac8d13cde6cec82c88 Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 23:21:48 +0000 Subject: [PATCH 3/5] Kernel: reject a host path, query or fragment when a port is set A host with a path, query or fragment cannot connect on the kernel path, because the kernel puts its API paths after the host. Before this commit, `host: 'h.io/path:8443'` also parsed as host `h.io/path` with embedded port 8443, which passed a broken host straight through. With an explicit `port` (DBSQLClient always supplies 443 by default) such a host now throws HiveDriverError before the embedded-port parse. Without a `port` the host is passed through unchanged, as before. Thrift is unchanged. A test in DBSQLClient.test.ts pins the current Thrift URL for a host with an embedded port (`https://h:8443:443/`), which node-fetch rejects, so the kernel fix does not alter Thrift. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 +- lib/kernel/KernelAuth.ts | 14 ++++++++------ tests/unit/DBSQLClient.test.ts | 21 +++++++++++++++++++++ tests/unit/kernel/connectionOptions.test.ts | 6 ++++++ 4 files changed, 36 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 623a1bd3..d5fc4ccf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased -- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, a falsy `port` (`0`, `NaN`, `''`, `null`, `false`) and an omitted port leave `host` unchanged; a numeric string is accepted; a port embedded in `host` wins over a different `port` (a warning is logged); an out-of-range port throws `HiveDriverError`. With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). +- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, a falsy `port` (`0`, `NaN`, `''`, `null`, `false`) and an omitted port leave `host` unchanged; a numeric string is accepted; a port embedded in `host` wins over a different `port` (a warning is logged); an out-of-range port throws `HiveDriverError`; a `host` with a path, query or fragment throws when a `port` is also set. With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). - The Thrift token federation exchange (`enableTokenFederation`) now goes through the connection's agent, so `proxy`, `checkServerCertificate`, `customCaCert` and `clientCert`/`clientKey` apply to it. Previously it connected directly with Node's default agent, so behind a required proxy the exchange failed and the driver fell back to the original token. (#611) - Kernel backend: a failed status poll while waiting for a statement (a `runAsync: true` statement, or a default execution that outlives the server's inline wait) and a failed result fetch in `hasMoreRows()` now throw the mapped driver error, such as `HiveDriverError: HTTP 500: Internal Server Error` once retries on HTTP 5xx run out, instead of a raw `Error` whose message is the internal `__databricks_error__:{...}` envelope. As in `fetchChunk()`, a failed fetch in `hasMoreRows()` also closes the statement. (#608) - Honor `cloudFetchConcurrentDownloads` when opening kernel sessions, with a default window of 16 and explicit chunk-setting overrides. Values other than positive safe integers raise `HiveDriverError` at session open. diff --git a/lib/kernel/KernelAuth.ts b/lib/kernel/KernelAuth.ts index 4c19353c..18261526 100644 --- a/lib/kernel/KernelAuth.ts +++ b/lib/kernel/KernelAuth.ts @@ -862,6 +862,14 @@ function buildKernelHostName(options: ConnectionOptions, logger?: IDBSQLLogger): const port = explicitPort(options.port); const scheme = /^https?:\/\//i.exec(options.host)?.[0] ?? ''; const authority = options.host.slice(scheme.length).replace(/\/$/, ''); + // A path, query or fragment in host never connected (the kernel puts its API paths after it), + // so with an explicit port it is rejected here, before any embedded-port parse can accept it. + if (port !== undefined && /[/?#]/.test(authority)) { + throw new HiveDriverError( + `kernel backend: host ${JSON.stringify(options.host)} has a path, query or fragment, so \`port\` ${port} ` + + 'cannot be applied to it. Use a bare host or host:port.', + ); + } const match = EMBEDDED_PORT_RE.exec(authority); const embedded = match ? Number(match[2]) : undefined; if (embedded !== undefined && (embedded < 1 || embedded > 65535)) { @@ -878,12 +886,6 @@ function buildKernelHostName(options: ConnectionOptions, logger?: IDBSQLLogger): // Fold a port into a bare host. `https: false` always folds a supplied port, while HTTPS // folds only a non-default one (443 is the default and adds nothing). if (embedded === undefined && port !== undefined && (options.https === false || port !== 443)) { - if (/[/?#]/.test(authority)) { - throw new HiveDriverError( - `kernel backend: host ${JSON.stringify(options.host)} has a path, query or fragment, so \`port\` ${port} ` + - 'cannot be added to it. Use a bare host.', - ); - } const hostPort = `${bracketIpv6(authority)}:${port}`; return options.https === false ? `http://${hostPort}` : `${scheme}${hostPort}`; } diff --git a/tests/unit/DBSQLClient.test.ts b/tests/unit/DBSQLClient.test.ts index 3638847a..3ed48b4c 100644 --- a/tests/unit/DBSQLClient.test.ts +++ b/tests/unit/DBSQLClient.test.ts @@ -234,6 +234,27 @@ describe('DBSQLClient.connect', () => { expect(connectionOptions.port).to.equal(3015); }); + it('keeps the Thrift URL for a host with an embedded port unchanged (current behavior, not changed by the kernel fix)', async () => { + const client = new DBSQLClient(); + + // Thrift appends options.port (443 by default) after the host, so an embedded port becomes + // `h:8443:443`, which node-fetch rejects as an invalid URL. This pins that current behavior. + const https = client['getConnectionOptions']({ ...connectOptions, host: 'h.databricks.com:8443', port: 443 }); + expect(https.host).to.equal('h.databricks.com:8443'); + expect(https.port).to.equal(443); + const httpsConnection = await new HttpConnection(https, client).getThriftConnection(); + expect(httpsConnection.url).to.equal('https://h.databricks.com:8443:443/'); + + const http = client['getConnectionOptions']({ + ...connectOptions, + host: 'localhost:3015', + port: 80, + https: false, + }); + const httpConnection = await new HttpConnection(http, client).getThriftConnection(); + expect(httpConnection.url).to.equal('http://localhost:3015:80/'); + }); + it('treats a falsy port as the default (443 over HTTPS, 80 over HTTP), as the kernel path does', async () => { const client = new DBSQLClient(); diff --git a/tests/unit/kernel/connectionOptions.test.ts b/tests/unit/kernel/connectionOptions.test.ts index 253e828b..9e0c54c1 100644 --- a/tests/unit/kernel/connectionOptions.test.ts +++ b/tests/unit/kernel/connectionOptions.test.ts @@ -200,8 +200,14 @@ describe('KernelAuth connection options — port folding', () => { ], ), [{ host: 'h.databricks.com/x', port: 8443 }, /has a path, query or fragment/], + [{ host: 'h.databricks.com/x', port: 443 }, /has a path, query or fragment/], // DBSQLClient always supplies 443 + [{ host: 'h.databricks.com/x/', port: 8443 }, /has a path, query or fragment/], + [{ host: 'https://h.databricks.com/x', port: 8443 }, /has a path, query or fragment/], [{ host: 'h.databricks.com?x', port: 8443 }, /has a path, query or fragment/], [{ host: 'h.databricks.com#x', port: 3015, https: false }, /has a path, query or fragment/], + // An embedded port after a path must not slip through the embedded-port parse. + [{ host: 'h.io/path:8443', port: 443 }, /has a path, query or fragment/], + [{ host: 'h.io/path:8443', port: 8443 }, /has a path, query or fragment/], [{ host: 'h.databricks.com:70000' }, /outside 1\.\.65535/], ]; it('rejects invalid ports and unfoldable hosts', () => { From 9b2e94d0a12594e7fd9bc600891cfde9d6a6fc86 Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 23:29:03 +0000 Subject: [PATCH 4/5] Kernel: reject a host path only when the port is folded or follows the path The path, query and fragment check from the previous commit rejected any explicit port, including the default 443 that DBSQLClient always supplies. That broke inputs that passed through on origin/main: `h/x` with no port or with port 443 over HTTPS was passed to the kernel verbatim before this change. The check now fires only when the port would be folded into the host (`https: false`, or a non-443 port over HTTPS) or when the host carries a port after the path (`h.io/path:8443`), which must not be read as port 8443. With no port, or with 443 over HTTPS and no embedded port, the host passes through unchanged. The check still runs before the embedded-port parse. Tests cover the pass-through cases (no port, port 443, https://host/x) and keep the rejection cases for the folded and embedded-after-path forms. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 +- lib/kernel/KernelAuth.ts | 12 +++++++++--- tests/unit/kernel/connectionOptions.test.ts | 7 +++++-- 3 files changed, 15 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d5fc4ccf..07fa5145 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased -- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, a falsy `port` (`0`, `NaN`, `''`, `null`, `false`) and an omitted port leave `host` unchanged; a numeric string is accepted; a port embedded in `host` wins over a different `port` (a warning is logged); an out-of-range port throws `HiveDriverError`; a `host` with a path, query or fragment throws when a `port` is also set. With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). +- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, a falsy `port` (`0`, `NaN`, `''`, `null`, `false`) and an omitted port leave `host` unchanged; a numeric string is accepted; a port embedded in `host` wins over a different `port` (a warning is logged); an out-of-range port throws `HiveDriverError`; a `host` with a path, query or fragment throws when the `port` is folded into it (`https: false`, or a non-443 `port` over HTTPS) or when `host` carries a port after the path. With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). - The Thrift token federation exchange (`enableTokenFederation`) now goes through the connection's agent, so `proxy`, `checkServerCertificate`, `customCaCert` and `clientCert`/`clientKey` apply to it. Previously it connected directly with Node's default agent, so behind a required proxy the exchange failed and the driver fell back to the original token. (#611) - Kernel backend: a failed status poll while waiting for a statement (a `runAsync: true` statement, or a default execution that outlives the server's inline wait) and a failed result fetch in `hasMoreRows()` now throw the mapped driver error, such as `HiveDriverError: HTTP 500: Internal Server Error` once retries on HTTP 5xx run out, instead of a raw `Error` whose message is the internal `__databricks_error__:{...}` envelope. As in `fetchChunk()`, a failed fetch in `hasMoreRows()` also closes the statement. (#608) - Honor `cloudFetchConcurrentDownloads` when opening kernel sessions, with a default window of 16 and explicit chunk-setting overrides. Values other than positive safe integers raise `HiveDriverError` at session open. diff --git a/lib/kernel/KernelAuth.ts b/lib/kernel/KernelAuth.ts index 18261526..a604f1d6 100644 --- a/lib/kernel/KernelAuth.ts +++ b/lib/kernel/KernelAuth.ts @@ -862,9 +862,15 @@ function buildKernelHostName(options: ConnectionOptions, logger?: IDBSQLLogger): const port = explicitPort(options.port); const scheme = /^https?:\/\//i.exec(options.host)?.[0] ?? ''; const authority = options.host.slice(scheme.length).replace(/\/$/, ''); - // A path, query or fragment in host never connected (the kernel puts its API paths after it), - // so with an explicit port it is rejected here, before any embedded-port parse can accept it. - if (port !== undefined && /[/?#]/.test(authority)) { + // A path, query or fragment in host cannot take a port: it is rejected when the port would be + // folded in (https: false, or HTTPS with a non-443 port) or when the host carries an embedded + // port after the path (`h.io/path:8443`), before that embedded port can be read. With no port, + // or with 443 over HTTPS and no embedded port, the host is passed through unchanged, as on origin/main. + if ( + port !== undefined && + /[/?#]/.test(authority) && + (options.https === false || port !== 443 || EMBEDDED_PORT_RE.test(authority)) + ) { throw new HiveDriverError( `kernel backend: host ${JSON.stringify(options.host)} has a path, query or fragment, so \`port\` ${port} ` + 'cannot be applied to it. Use a bare host or host:port.', diff --git a/tests/unit/kernel/connectionOptions.test.ts b/tests/unit/kernel/connectionOptions.test.ts index 9e0c54c1..b32210f8 100644 --- a/tests/unit/kernel/connectionOptions.test.ts +++ b/tests/unit/kernel/connectionOptions.test.ts @@ -168,7 +168,11 @@ describe('KernelAuth connection options — port folding', () => { [{ host: 'https://h.databricks.com/', port: 8443 }, 'https://h.databricks.com:8443'], [{ host: 'h.databricks.com:8443/' }, 'h.databricks.com:8443'], [{ host: 'h.databricks.com/' }, 'h.databricks.com'], - [{ host: 'h.databricks.com/x' }, 'h.databricks.com/x'], // no port: unchanged, as before + // A path host with no port, or with 443 over HTTPS, is not folded: passed through unchanged, as before. + [{ host: 'h.databricks.com/x' }, 'h.databricks.com/x'], + [{ host: 'h.databricks.com/x', port: 443 }, 'h.databricks.com/x'], + [{ host: 'https://h.databricks.com/x', port: 443 }, 'https://h.databricks.com/x'], + [{ host: 'h.databricks.com/x', port: 443, https: true }, 'h.databricks.com/x'], [{ host: 'h.databricks.com', port: 3015, https: false }, 'http://h.databricks.com:3015'], [{ host: 'localhost', port: 80, https: false }, 'http://localhost:80'], [{ host: 'localhost', port: '3015', https: false }, 'http://localhost:3015'], @@ -200,7 +204,6 @@ describe('KernelAuth connection options — port folding', () => { ], ), [{ host: 'h.databricks.com/x', port: 8443 }, /has a path, query or fragment/], - [{ host: 'h.databricks.com/x', port: 443 }, /has a path, query or fragment/], // DBSQLClient always supplies 443 [{ host: 'h.databricks.com/x/', port: 8443 }, /has a path, query or fragment/], [{ host: 'https://h.databricks.com/x', port: 8443 }, /has a path, query or fragment/], [{ host: 'h.databricks.com?x', port: 8443 }, /has a path, query or fragment/], From 468af3a0ae898021838b48d069d129125e431339 Mon Sep 17 00:00:00 2001 From: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> Date: Thu, 19 Nov 2026 23:35:57 +0000 Subject: [PATCH 5/5] Kernel: pass a path host through unless a folded port applies The path, query and fragment check now fires only when the explicit port is folded into the host (https: false, or a non-443 port over HTTPS). A host with no folded port passes through unchanged, as on origin/main, including a host with a port after the path (h.io/path:8443), which base passed to the kernel verbatim. The check still runs before the embedded-port parse. Tests keep the rejection cases for the folded form and add pass-through cases for no port, port 443, and an embedded port after a path. Co-authored-by: Isaac Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com> --- CHANGELOG.md | 2 +- lib/kernel/KernelAuth.ts | 13 ++++--------- tests/unit/kernel/connectionOptions.test.ts | 8 ++++++-- 3 files changed, 11 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 07fa5145..f4f9d5d9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Unreleased -- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, a falsy `port` (`0`, `NaN`, `''`, `null`, `false`) and an omitted port leave `host` unchanged; a numeric string is accepted; a port embedded in `host` wins over a different `port` (a warning is logged); an out-of-range port throws `HiveDriverError`; a `host` with a path, query or fragment throws when the `port` is folded into it (`https: false`, or a non-443 `port` over HTTPS) or when `host` carries a port after the path. With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). +- Kernel backend: a custom `port` is honored over HTTPS, as on Thrift. It was dropped unless `https: false` was set. A non-default port folds into `host` (bare IPv6 hosts are bracketed); `port: 443`, a falsy `port` (`0`, `NaN`, `''`, `null`, `false`) and an omitted port leave `host` unchanged; a numeric string is accepted; a port embedded in `host` wins over a different `port` (a warning is logged); an out-of-range port throws `HiveDriverError`; a `host` with a path, query or fragment throws when the `port` is folded into it (`https: false`, or a non-443 `port` over HTTPS). With OAuth and a non-default port, the kernel reaches its OAuth endpoints at the same `host:port`, while Thrift keeps OAuth on the bare host (documented, not guarded). - The Thrift token federation exchange (`enableTokenFederation`) now goes through the connection's agent, so `proxy`, `checkServerCertificate`, `customCaCert` and `clientCert`/`clientKey` apply to it. Previously it connected directly with Node's default agent, so behind a required proxy the exchange failed and the driver fell back to the original token. (#611) - Kernel backend: a failed status poll while waiting for a statement (a `runAsync: true` statement, or a default execution that outlives the server's inline wait) and a failed result fetch in `hasMoreRows()` now throw the mapped driver error, such as `HiveDriverError: HTTP 500: Internal Server Error` once retries on HTTP 5xx run out, instead of a raw `Error` whose message is the internal `__databricks_error__:{...}` envelope. As in `fetchChunk()`, a failed fetch in `hasMoreRows()` also closes the statement. (#608) - Honor `cloudFetchConcurrentDownloads` when opening kernel sessions, with a default window of 16 and explicit chunk-setting overrides. Values other than positive safe integers raise `HiveDriverError` at session open. diff --git a/lib/kernel/KernelAuth.ts b/lib/kernel/KernelAuth.ts index a604f1d6..ade0e495 100644 --- a/lib/kernel/KernelAuth.ts +++ b/lib/kernel/KernelAuth.ts @@ -862,15 +862,10 @@ function buildKernelHostName(options: ConnectionOptions, logger?: IDBSQLLogger): const port = explicitPort(options.port); const scheme = /^https?:\/\//i.exec(options.host)?.[0] ?? ''; const authority = options.host.slice(scheme.length).replace(/\/$/, ''); - // A path, query or fragment in host cannot take a port: it is rejected when the port would be - // folded in (https: false, or HTTPS with a non-443 port) or when the host carries an embedded - // port after the path (`h.io/path:8443`), before that embedded port can be read. With no port, - // or with 443 over HTTPS and no embedded port, the host is passed through unchanged, as on origin/main. - if ( - port !== undefined && - /[/?#]/.test(authority) && - (options.https === false || port !== 443 || EMBEDDED_PORT_RE.test(authority)) - ) { + // A path, query or fragment in host cannot take a folded port: it is rejected when the port would + // be folded in (https: false, or HTTPS with a non-443 port). With no port, or with 443 over HTTPS, + // the host is passed through unchanged, as on origin/main (including `h.io/path:8443`). + if (port !== undefined && /[/?#]/.test(authority) && (options.https === false || port !== 443)) { throw new HiveDriverError( `kernel backend: host ${JSON.stringify(options.host)} has a path, query or fragment, so \`port\` ${port} ` + 'cannot be applied to it. Use a bare host or host:port.', diff --git a/tests/unit/kernel/connectionOptions.test.ts b/tests/unit/kernel/connectionOptions.test.ts index b32210f8..42aeb1f1 100644 --- a/tests/unit/kernel/connectionOptions.test.ts +++ b/tests/unit/kernel/connectionOptions.test.ts @@ -173,6 +173,9 @@ describe('KernelAuth connection options — port folding', () => { [{ host: 'h.databricks.com/x', port: 443 }, 'h.databricks.com/x'], [{ host: 'https://h.databricks.com/x', port: 443 }, 'https://h.databricks.com/x'], [{ host: 'h.databricks.com/x', port: 443, https: true }, 'h.databricks.com/x'], + // An embedded port after a path, with no folded port, passes through as on origin/main. + [{ host: 'h.io/path:8443' }, 'h.io/path:8443'], + [{ host: 'h.io/path:8443', port: 443 }, 'h.io/path:8443'], [{ host: 'h.databricks.com', port: 3015, https: false }, 'http://h.databricks.com:3015'], [{ host: 'localhost', port: 80, https: false }, 'http://localhost:80'], [{ host: 'localhost', port: '3015', https: false }, 'http://localhost:3015'], @@ -208,9 +211,10 @@ describe('KernelAuth connection options — port folding', () => { [{ host: 'https://h.databricks.com/x', port: 8443 }, /has a path, query or fragment/], [{ host: 'h.databricks.com?x', port: 8443 }, /has a path, query or fragment/], [{ host: 'h.databricks.com#x', port: 3015, https: false }, /has a path, query or fragment/], - // An embedded port after a path must not slip through the embedded-port parse. - [{ host: 'h.io/path:8443', port: 443 }, /has a path, query or fragment/], + // A folded port (explicit non-443, or https: false) on a path host is rejected, including + // when the host also carries an embedded port after the path. [{ host: 'h.io/path:8443', port: 8443 }, /has a path, query or fragment/], + [{ host: 'h.io/path:8443', port: 443, https: false }, /has a path, query or fragment/], [{ host: 'h.databricks.com:70000' }, /outside 1\.\.65535/], ]; it('rejects invalid ports and unfoldable hosts', () => {