Repository navigation
Complete Volume file operations through the kernel backend - #1057
aakash-saravanan-db wants to merge 42 commits into
Conversation
There was a problem hiding this comment.
Verdict: 1 High · 1 Medium · 1 Low
Needs changes. The main blocker is that the code relies on new kernel APIs while the dependency floor still allows older wheels, which would break every kernel query, not only Volume operations. There is also a KeyError crash when the best-effort canceller is missing.
Other findings
- 🔵 Low — This PR deletes the kernel e2e tests for staging and adds no replacements. All new coverage uses mocks that invent
is_volume_operation,complete_volume_operationreturn strings ("needs_input_stream","completed","not_volume") and exception attributes (local_os_error,http_status,local_path). Nothing in CI checks that the real kernel binding actually has these names and values. Please add kernel e2e cases, such as PUT→GET→REMOVE round trips (file andinput_stream) and a rejected symlink outside the allowed root, so a mismatch with the kernel contract fails the build.
| # Canceller is best-effort; never block execute on it. | ||
| pass | ||
| executed = stmt.execute() | ||
| if executed.is_volume_operation: |
There was a problem hiding this comment.
🟠 High — pyproject.toml:73 still allows databricks-sql-kernel ^1.4.0, but the code now calls three kernel APIs that don't exist yet: ExecutedStatement.is_volume_operation (read here on every sync execute), kernel_handle.is_volume_operation (read in KernelResultSet.__init__ for every result, including metadata ResultStreams), and fetch_result() in place of await_result() for async results.
If the resolver installs any wheel in the allowed range that lacks them, this isn't limited to Volume ops. Every execute(), every metadata call and every get_async_execution_result() raises AttributeError, so ordinary SELECT 1 breaks on use_kernel=True. The PR description says the floor "has not been selected". This should block merge: bump the floor in the same change. Until then, reading is_volume_operation with getattr(..., False) would at least keep non-volume queries working.
There was a problem hiding this comment.
The optional-canceller defect is corrected at ab1efce1: registration uses .get, the Statement remains retained, and cleanup skips absent cancellers. All 276 affected tests pass; final native live checks preserve lazy BinaryIO and one-read empty/nonempty streams.
The new Volume marker/completion API dependency is a valid merge prerequisite. The source build pins compatible 22f96fac; a compatible published native version/floor and installed-package proof remain explicitly open. This PR is not merge-ready, and no old-native fallback or invented release version was added.
The fetch_result subclaim is separate: released 1.4.0 exposes it, but the substitution is removed from this PR. Merged native #451 provides terminal-response reuse with the original await_result boundary. Actual public-cursor owner and reader/refetch tests show identical await/fetch requests, rows and metadata on the selected native; redundant #1056 is superseded without merge.
There was a problem hiding this comment.
Thanks for confirming the Volume-marker dependency is a merge prerequisite, and for dropping the fetch_result() substitution. That part of the finding is resolved. The core issue is still there, though. pyproject.toml:73 still says databricks-sql-kernel ^1.4.0. Line 641 still reads executed.is_volume_operation with no guard. KernelResultSet.__init__ still reads kernel_handle.is_volume_operation the same way. So if a published wheel in that range lacks the marker, ordinary sync executes and metadata calls fail with AttributeError before the code ever checks for a Volume op. A source build pinned to 22f96fac doesn't limit what the resolver installs. Leaving this open until the floor is raised to the first published wheel that ships the marker, or until both reads are guarded so non-volume queries keep working.
There was a problem hiding this comment.
Verdict: 1 High · 1 Medium · 2 Nit
Needs changes: 1 high, 1 medium and 2 low findings. The high one: the new is_volume_operation attribute is read unconditionally on every execute, but the kernel floor is still ^1.4.0, so released wheels would break all kernel queries. The medium one: any kernel error with an HTTP status is turned into a generic staging OperationalError. Follow-up already noted in the PR: Windows and installed released-package validation are still open.
Other findings
- ⚪ Nit — The new ordering (backend
close_commandfirst, then_kernel_handle.close()) now applies to every kernel result set, not only Volume ones. For async results,close_commandcloses the retained owning handle and the parentStatement(client.py ~800-820) while the result stream taken from them is still open. Before this change the stream was closed first. Any error from closing the stream afterwards is logged and swallowed, but this changes the teardown order for all async results to work around a lock that only Volume transfers hold. Consider cancelling only the Volume canceller first and keeping the old order for everything else. - ⚪ Nit — Several new cancellation paths have no unit tests:
close_sessionnow cancels all in-flight sync cancellers and retained Volume cancellers, andclose_commandcancels a pending Volume canceller (client.py:784-790). The only Volume backend test coverscancel_commandandfinish_volume_operation.test_volume_close_signals_backend_before_native_lockuses a mocked backend, so it doesn't exercise the realclose_commandlogic. Add tests that (a)close_commandon a registered Volume id callscancel()and removes the entry, and (b)close_sessioncancels and clears both maps, even whencancel()raises.
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 High
Needs a dependency-floor fix: 1 medium issue. The completion, cancel and close lifecycle for sync and async Volume operations looks consistent. Nit: the new error-message strings in _complete_volume_operation go past the 100-character line limit.
| results_queue=None, | ||
| description=description_from_arrow_schema(schema), | ||
| is_staging_operation=False, | ||
| is_staging_operation=kernel_handle.is_volume_operation, |
There was a problem hiding this comment.
🟠 High — KernelResultSet.__init__ reads kernel_handle.is_volume_operation unguarded, but pyproject still allows ^1.4.0. A kernel wheel without the Volume API would raise AttributeError on every query and metadata call. Raise the dependency floor to the release that adds the Volume API, or read the attribute with getattr(..., False).
🔀 Found by both reviewers · confirmed by the judge
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 High
1 high concern. Completion, cancellation, and error mapping work as described and match Thrift's allowed-path and stream semantics. However, the new kernel attributes are read on every query, so this PR depends on the pending dependency-floor bump to avoid breaking all kernel queries.
| results_queue=None, | ||
| description=description_from_arrow_schema(schema), | ||
| is_staging_operation=False, | ||
| is_staging_operation=kernel_handle.is_volume_operation, |
There was a problem hiding this comment.
🟠 High — KernelResultSet.__init__ reads kernel_handle.is_volume_operation for every result, but pyproject.toml still allows ^1.4.0 and the lock pins 1.4.0. The PR only bumps KERNEL_REV, which affects CI source, not the released wheel. An installed wheel without this API would raise AttributeError on every kernel query. Raise the floor and lock to the released wheel that ships is_volume_operation/VolumeCompletion, or read it with getattr(..., False).
🔀 Found by both reviewers · confirmed by the judge
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 High
Needs changes: 1 high concern. The completion, cancellation and error-mapping flow looks correct and has tests. But because the packaging floor is not bumped, every kernel query breaks on wheels that are currently allowed.
| pass | ||
| executed = stmt.execute() | ||
| command_id = CommandId.from_sea_statement_id(executed.statement_id) | ||
| if executed.is_volume_operation: |
There was a problem hiding this comment.
🟠 High — The PR reads executed.is_volume_operation and kernel_handle.is_volume_operation unconditionally, but pyproject.toml and poetry.lock still allow kernel 1.4.0. That release almost certainly lacks the Volume API, since the PR only bumps KERNEL_REV (CI source). Installed wheels could then fail every query with AttributeError. Raise the floor and lock to the release that ships the Volume API, or read the attribute with getattr(..., False).
🔀 Found by both reviewers · confirmed by the judge
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 High
One high-severity blocker: the kernel dependency floor still allows wheels that lack the Volume API. The completion, cancellation and error-mapping logic matches the Thrift Volume path, and the unit tests cover the main outcomes.
| results_queue=None, | ||
| description=description_from_arrow_schema(schema), | ||
| is_staging_operation=False, | ||
| is_staging_operation=kernel_handle.is_volume_operation, |
There was a problem hiding this comment.
🟠 High — The kernel result set and execute_command now read is_volume_operation unconditionally, but pyproject still allows ^1.4.0. A wheel without the Volume API would raise AttributeError on every kernel query. Raise the floor to the release that adds the Volume API and update the lock file.
🔀 Found by both reviewers · confirmed by the judge
Feature or gap
Kernel-backed cursors need to complete server-marked Volume transfers through the Python staging API.
What this PR addresses
Complete file PUT, GET, REMOVE and BinaryIO PUT through the native core. Keep the mutable statement owner and detached canceller available during synchronous transfer, consume instruction rows on completion, and preserve Python filesystem and HTTP exception mapping. Async execution completes through its retained result stream.
Dependencies
Requires the kernel Volume APIs in databricks/databricks-sql-kernel#465 and a compatible coordinated native release. The package floor and lock must select that release, with clean installed-driver validation before merge.
Kernel version
The compatible kernel release has not been selected. When it is available, update the dependency floor and lockfile in this PR and validate a clean installation before merge.
KERNEL_REVselects source for local and CI builds only.Tests and validation
The focused client and result-set suites pass: 292 tests. Local validation uses source-built native inputs; package installation and scheduled activation remain separate gates.