Repository navigation
Conversation
The kernel backend (use_kernel=True) ignored the Thrift-style _port and _connection_uri overrides. Resolve both connector-side onto the kernel's single host and http_path, with the Thrift rules: - _port (1..65535; 0 or None mean the default) folds into server_hostname as host:port. The default 443 adds nothing, IPv6 literals are bracketed, and an embedded port equal to _port is accepted. An empty server_hostname with _port raises ValueError. - _connection_uri (http(s)://host[:port]/path?query) replaces host, port, path and query, and wins over _port. Scheme-less, non-http(s), path-less and credential-bearing URIs raise ValueError. URI-only use works. - With OAuth (U2M, Azure service principal, JWT key file, or M2M client id and secret), a _connection_uri whose origin differs from https://server_hostname raises before a kernel session opens. PAT is not affected. A non-default _port with OAuth is not guarded: the token exchange goes to the same host:port, which is documented. - The SPOG x-databricks-org-id header follows the resolved path. Tests cover the resolver tables, the OAuth origin guard, the SPOG re-derivation, and a real-wheel loopback dial proving the kernel reaches the requested port over TLS for _port and for _connection_uri. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
The helper returned Optional[str] for the resolved path, but KernelDatabricksClient takes a str, so mypy fails in check-types on every supported Python. Session.__init__ already types http_path as str, so this is an annotation-only change with no runtime effect. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 2 Low
Looks good: two low concerns. Resolver, OAuth origin guard and SPOG re-derivation match Thrift's precedence and have thorough table tests. Nit: a few new lines (the Conflicting-ports f-string, the _split_host and _uri_host_and_path docstrings) exceed the 100-char limit in CONTRIBUTING.md.
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 2 Low
Looks good, with 1 low concern. The resolver, the OAuth origin guard and the SPOG re-derivation match the auth bridge's routing and are well covered by tests. Nit: a few new lines go past the 100-char limit (the Conflicting ports f-string and the _split_host/_uri_host_and_path docstrings), and the AC-P4 ticket reference in the docstring.
This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.
A kernel connection without _port or _connection_uri again passes server_hostname through unchanged, so odd hosts no longer raise on the default path. The OAuth origin guard now skips connections the kernel auth bridge routes to PAT: an access_token with no JWT or client-secret M2M credentials, even when auth_type is databricks-oauth. azure-sp-m2m and M2M/JWT stay guarded. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Looks good, with 1 low concern. The resolver, the OAuth predicate (which matches the auth bridge's routing) and the SPOG re-derivation all check out. Nits: session.py:135 and the _split_host/_uri_host_and_path docstrings go past the 100-char limit, and test_session.py imports databricks.sql twice.
Wraps the origin-guard condition to the repo's black line length (check-linting). The guard no longer treats a custom credentials_provider as OAuth: the kernel auth bridge rejects that provider with NotSupportedError, so no OAuth origin is involved. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
| # M2M credentials to PAT, even for auth_type='databricks-oauth'; azure-sp-m2m | ||
| # is resolved before that branch and stays OAuth. A custom credentials_provider | ||
| # is rejected by the bridge, so it is not an OAuth origin here. | ||
| routed_to_pat = ( |
There was a problem hiding this comment.
The kernel wraps every auth provider, PAT included, in FederationProvider using the connection host (session.rs:1394, host = the _connection_uri origin here). So a foreign-issuer JWT passed as access_token (Azure AD, BYOT, WIF) is POSTed to {_connection_uri host}/oidc/v1/token. Thrift does that exchange at server_hostname (auth.py:70) and the URI host only sees the minted token. _check_oauth_origin classifies every access_token as PAT, so this passes the guard with or without identity_federation_client_id. Suggest treating an access_token that decodes as a JWT with a non-server_hostname issuer as OAuth here (decode_token / is_same_host in auth_utils.py), plus a test.
A port embedded in server_hostname now wins over a differing _port, with a warning, as on main, where the kernel connected to the embedded port. A falsy _port (None, 0, '', False) means the default, as on Thrift (port or 443). A trailing '/' is trimmed before port handling. A truthy invalid _port still raises, as Thrift fails on it (LocationParseError). Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Low
Looks good, no blocking issues. The OAuth origin guard's PAT-routing predicate matches the precedence in auth_bridge.kernel_auth_kwargs. Nits: an embedded port plus a path with _port set raises a misleading "malformed IPv6" error, and test_resolved_kernel_host_and_path_reach_kernel_client never closes its connections.
This overview is Claude Opus 5.5's initial take; GPT-6.1 Sol added findings it doesn't mention — see the findings below.
…ernel port rules Check the host for a path, query or fragment before the port is parsed, so 'h.io/path:8443' with _port cannot read as port 8443. Only explicit _port is affected; a host without _port passes through unchanged, as on main. Add Thrift tests pinning that host and _port reach ThriftDatabricksClient unchanged. Update CONNECTION_PARAMETERS.md: _port and _connection_uri are now kernel options, with the embedded-port and falsy rules and the OAuth origin rule. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Replace the _HOSTPORT_RE regex, the _port regex and the manual bracketing in the kernel host folding with urllib.parse.urlsplit, int() and an ipaddress pre-check for bare IPv6 literals. The original host string still passes through unchanged unless a port is folded in; folded IPv6 hosts are bracketed.
_check_oauth_origin treats an access_token that decodes as a JWT whose issuer is not server_hostname as OAuth. The kernel wraps every provider with the connection host, so such a token would be exchanged at the _connection_uri origin, while Thrift exchanges it at server_hostname. A mismatched origin now raises before any request. PATs and same-host issuers are unaffected.
The port-conflict warning is skipped when _port is the default 443, matching Node. Non-default ports still warn on a mismatch.
Behavior-table tests in test_session.py pass unchanged. One warning test changed: test_embedded_port_wins_over_differing_explicit_port lost its ('h.io:8443', 443) row, which asserted the 443 warning. That case moved to test_embedded_port_with_default_443_does_not_warn under the new rule.
Known differences from bc7c09e: the OAuth origin check now compares the parsed hostname, so a server_hostname such as 'h.io/x' or 'user@h' with a matching _connection_uri origin passes where it was previously refused. The error text for 'h:70000', 'h:abc' and '[::1' is now one generic malformed-or-out-of-range message. 'user:pass@h' with _port is still refused.
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
_is_bare_ipv6 now returns early unless the text has at least two colons and no leading bracket. ip_address raised a ValueError for every host name and host:port string, which made the folding path about 2x slower than bc7c09e. A bare IPv6 literal always has two or more colons, so behavior is unchanged; the session tests and the realkernel loopback tests pass. Microbenchmark, median us per call (20000 calls x 7 repeats): - bare host, _port 8443: 1.305 base, 1.261 new - host:port, _port 8443: 1.364 base, 1.393 new (+2%) - IPv6 bracketed, _port 8443: 1.340 base, 1.355 new (+1%) - IPv6 bare, _port 8443: 1.413 base, 2.642 new (+87%, the successful ip_address parse) Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
Remove _is_bare_ipv6 and the ipaddress import. Bare IPv6 hosts such as ::1 never worked on the kernel path: the kernel prepends https:// and its URL parser rejects an unbracketed IPv6 literal. IPv6 literals must be bracketed ([::1]).
With _port set, urlsplit reads everything after the first colon of a bare IPv6 literal as the port. That text always contains another colon, so the integer parse fails and the existing "malformed IPv6 literal or port" ValueError is raised before any request. This includes _port=443. Without _port the host still passes through unchanged, as on main.
Tests: the ("::1", 8443) -> "[::1]:8443" row is removed. ::1 with _port 8443 or 443, and 2001:db8::1 with _port 8443, are now in the reject table.
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Cathleen Yan <58714163+cathleeny@users.noreply.github.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
✅ No issues identified.
🔀 1 finding considered and dismissed by the judge
- 🟡 Medium
src/databricks/sql/session.py:240— An access_token combined with identity_federation_client_id bypasses the origin guard when its issuer matches server_hostname, although the bridge forwards that
Judge's evidence:if routed_to_pat or not oauth:
What type of PR is this?
Description
Honors the Thrift-style
_portand_connection_urioverrides on the kernel backend (use_kernel=True). The kernel takes one host and onehttp_path, so both are resolved connector-side insession.py(_kernel_host_and_path). No kernel change.Host and port are parsed with the standard library:
urllib.parse.urlsplitforhost[:port]andint()for the port. There is no hand-written host regex.server_hostnamepasses through unchanged unless a port is folded in. IPv6 literals must be bracketed ([::1]).Port precedence, the same in the Python, Node and Go drivers:
A port embedded in
server_hostnamealways wins._portapplies only to a bare host. If a non-default_portdiffers from the embedded port, a WARNING names both ports, and the connection uses the embedded port. It does not raise. An explicit_port=443, the default, never warns.A falsy
_port(None,0,'',False) means the default. No:443is added.A trailing
/onserver_hostnameis trimmed before port handling.A truthy invalid
_port(non-numeric, or outside 1..65535) raisesValueError, because Thrift also fails on it (see Base behavior).An explicit
_portwith aserver_hostnamethat has a path, query or fragment raisesValueError. The check runs before the port is parsed, soh.io/path:8443is rejected rather than read as port 8443. The kernel builds its URL from the host, so these configs never connected. A host without_portpasses through unchanged, as on main.In this driver, an explicit
_portwith a malformedserver_hostnameraisesValueError. Malformed means a trailing:, a broken or unbracketed IPv6 literal, a non-numeric or out-of-range embedded port, oruser:password@credentials. The error message redacts the host. A host with user info but no password (user@h) is still folded, as before._connection_uri(http(s)://host[:port]/path?query) sets host, port, path and query, and wins over_port, as on Thrift. Scheme-less, non-http(s), path-less and credential-bearing URIs raiseValueError. URI-only use (emptyserver_hostnameandhttp_path) works.Some auth types count as OAuth for this check:
access_tokenthat is a JWT from an issuer other thanserver_hostname.With any of these, a
_connection_uriwhose origin (scheme, host, port) differs fromhttps://server_hostnameraises before a kernel session opens. PAT is unaffected, and so is a JWT whose issuer isserver_hostname.The SPOG
x-databricks-org-idheader follows the resolved path when_connection_urichanges it.Thrift is unchanged.
session.pypassesserver_hostnameand_porttoThriftDatabricksClientverbatim, andthrift_backend.pyappliesport or 443. A unit test pins this for the same configs.CONNECTION_PARAMETERS.mdnow lists_portand_connection_urias Kernel options, with the rules above.Base behavior (origin/main)
session.pypassesserver_hostnameandhttp_pathtoKernelDatabricksClientverbatim. The kernel path ignores_portentirely. An embedded host port passes through, so the kernel connects to it, which is why the embedded port wins.client.pypassesport=kwargs.get("_port", 443)only to Thrift. There is no_connection_urion main.thrift_backend.pylines 170-176):port = port or 443, then"{host}:{port}/{path}". A falsy_porttherefore means 443. A non-numeric, out-of-range, negative or boolean_portbuilds a URL that urllib3 rejects withLocationParseError(checked with the connector's urllib3 forabc,70000,-1,True,8443.5).Foreign-issuer JWT as access_token
Reviewer thread (
session.py,_check_oauth_origin): the kernel wraps every auth provider in FederationProvider using the connection host, PAT included. A JWT from a foreign issuer passed asaccess_tokenwould therefore be exchanged at the_connection_uriorigin, while Thrift exchanges it atserver_hostname. The guard now treats such a token as OAuth. It reuses the connector'sdecode_tokenandis_same_hosthelpers, the same ones the Thrift token-federation path uses. A mismatched origin raises before any request.Differences from the previous revision of this PR
Conflicting ports). It now connects to the embedded port with a warning, and_port=443no longer warns at all._port('') andFalseused to raise. They now mean the default._portwith a host that has a path, query or fragment used to pass through verbatim, and the kernel then built a URL with the path in the authority. It now raises."0"(a string) as_portis treated as the default. Thrift would dial port 0. This is a relaxation.urlsplit. The behavior tables are unchanged. Two side effects:server_hostnamesuch ash.io/xoruser@hwith a matching_connection_uriorigin now passes where it was refused.h:70000,h:abcand[::1now share one malformed-or-out-of-range message.access_tokenis now subject to the OAuth origin check (above)._portused to be bracketed and folded (::1became[::1]:8443). It now raises; bracket it instead. Bare IPv6 never worked on the kernel path on main, because the kernel's URL parser rejects it. Without_port, the host still passes through unchanged.Overlap with databricks-sql-python#915
#915 is another engineer's open PR. It changes the same function (
_kernel_host_and_path) for the same feature. This branch is a self-contained change onmain(b099dd6), with no commits from #915. Only one of the two should merge.This PR also aligns validation with Thrift. It rejects inputs Thrift rejects: non-numeric or out-of-range ports, and path-less or scheme-less URIs. #915 defaulted scheme-less URIs to https and kept the original path. Coordinating with #915's owner is a human step.
How is this tested?
Unit tests
Other
Unit (
tests/unit,-m "not realkernel"): every test file run in its own process passes (test_init_file.pyneeds the repo as cwd and passes from there). Runningtest_session.pyandtest_kernel_client.pyin one process fails the real-wheel tests, because the latter installs a stub kernel module.tests/unit/test_session.py: 196 pass. The table cases cover:user:password@and trailing-:rejections with_port, theurlsplitedge cases (case, user info, broken IPv6, trailing/), and the scheme cases;caplog) and the 443 no-warning case;access_tokencases;test_thrift_receives_host_and_port_unchanged).tests/unit/test_kernel_client.py: 282 pass.Realkernel (
-m realkernel,databricks-sql-kernel==1.4.0): 5 pass. The loopback tests aretest_kernel_dials_over_tls(bare host with_port, and URI-only) andtest_kernel_embedded_port_wins_over_explicit_port(server_hostname127.0.0.1:P1with_port=P2). The TCP listener on P1 receives the TLS handshake (first byte0x16), and the listener on P2 receives no connection.black --check src: clean.Microbenchmark of the host resolver, median per call: within 1-3% of the regex version for bare hosts,
host:portand bracketed IPv6.Live (kernel path, DBSQL
417ec0c098c5d0b6and Reyden0000000000e01f56, at f242c0f; the final commit only changes bare-IPv6 handling, which no live case uses): explicit_port443,_connection_urioverriding_port=1, and URI-only all pass on both warehouses. Thrift passes on DBSQL. The Reyden Thrift rows showunexpected-transport, the tracked baseline exception.Not covered live: a host with an embedded
:443(the probe has no such case).This pull request and its description were written by Isaac.