Skip to content

Commit 4c9a77e

Browse files
committed
Manage object lifetime during offload with deferred releases
Removed use of host_task that took the GIL and could free a USM allocation, so the task called sycl::free and destroyed a sycl::context inside the host_task. This is forbidden per the SYCL spec, and made dpctl unbuildable against AdaptiveCpp, which does not implement host_task. The releases are now deferred instead via a polling thread. Deprecates SyclQueue._submit_keep_args_alive in favor of SyclQueue.keep_args_alive, intended as a public method for keeping alive these Python objects in a way free of host_task. Also deprecates methods add_event_pair, num_host_task_events, and host_task_events on the SequentialOrderManager, while adding add_event, add_cleanup_event, and num_cleanup_events. _sycl_queue.pyx also gains an atexit hook to prevent crashes at process teardown.
1 parent 0a2b834 commit 4c9a77e

21 files changed

Lines changed: 1452 additions & 304 deletions

‎CHANGELOG.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77
## [dev] - XXX. XX, XXXX
88

99
### Added
10+
* Added `dpctl.SyclQueue.keep_args_alive` method, which keeps Python objects alive until the work on that queue using them completes [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
11+
* Added `add_event` and `add_cleanup_event` methods and `cleanup_events` and `num_cleanup_events` properties to the order manager, for tracking the events that gate the release of objects used by offloaded tasks [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1012

1113
### Changed
14+
* Object lifetime during offload is now managed by a list of releases deferred until the events that gate them have completed, rather than by a `host_task`. `dpctl` keeps one thread for looking over the list and running the releases that have come due, which takes the GIL to do so and lets a release call into the interpreter and free USM allocations [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
15+
* The event returned by `dpctl.SyclQueue.keep_args_alive`, `dpctl::utils::keep_args_alive` and `dpctl.SyclQueue._submit_keep_args_alive` is now for an empty kernel that gates the deferred release rather than for a `host_task` that performs it, so its completion means that the objects are no longer in use rather than that they have been released [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1216

1317
### Deprecated
1418

@@ -17,6 +21,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1721

1822
### Fixed
1923
* Removed stale `select_host_device` and `has_host_device` names from `dpctl.__all__`, which made `from dpctl import *` fail with `AttributeError` [gh-2410](https://github.com/IntelPython/dpctl/pull/2410)
24+
* A USM allocation that fails now drops the references held for offloaded tasks that have completed, and is attempted once more if that released anything, so that memory only waiting to be given up is not reported as unavailable [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
2025

2126
### Maintenance
2227
* Removed the `-fno-sycl-id-queries-fit-in-int` compile option, which none of the `dpctl` extensions need since `dpctl.tensor` moved to `dpnp` [gh-2413](https://github.com/IntelPython/dpctl/pull/2413)
@@ -66,6 +71,8 @@ This release extends the SYCL interface exposed by `dpctl` in several directions
6671
* Deprecated `dpctl.program.create_program_from_source` and `dpctl.program.create_program_from_spirv` in favor of `dpctl.compiler.create_kernel_bundle_from_source` and `dpctl.compiler.create_kernel_bundle_from_spirv`; the old functions are kept as aliases that emit a `DeprecationWarning` [gh-2291](https://github.com/IntelPython/dpctl/pull/2291), [gh-2317](https://github.com/IntelPython/dpctl/pull/2317)
6772
* Deprecated `dpctl.program` submodule in favor of `dpctl.compiler`, which provides a better description of the purpose of the submodule in exposing DPC++ compilation-related functionality [gh-2317](https://github.com/IntelPython/dpctl/pull/2317)
6873
* Deprecated `DPCTL_ENABLE_L0_PROGRAM_CREATION` CMake option in favor of `DPCTL_ENABLE_L0_KERNEL_BUNDLE_CREATION` [gh-2317](https://github.com/IntelPython/dpctl/pull/2317)
74+
* Deprecated `dpctl.SyclQueue._submit_keep_args_alive` in favor of `dpctl.SyclQueue.keep_args_alive`, which it now calls and which behaves identically [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
75+
* Deprecated the order manager's `add_event_pair`, `host_task_events` and `num_host_task_events`, as `host_task` is no longer used for managing object lifetimes, in favor of `add_event`, `add_cleanup_event`, `cleanup_events` and `num_cleanup_events` [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
6976

7077
### Removed
7178
* Removed unused `platform_typeid_lookup` from `dpctl4pybind11.hpp` [gh-2286](https://github.com/IntelPython/dpctl/pull/2286)

‎docs/doc_sources/api_reference/dpctl/utils.rst‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,23 @@
1515

1616
Thread-local object mapping each :class:`dpctl.SyclQueue` to an order
1717
manager, used to ensure sequential ordering of offloaded tasks.
18+
19+
Record submitted tasks with ``add_event`` and use ``submitted_events``
20+
as the dependency list of subsequent submissions. To keep Python objects
21+
referenced by a task alive until it completes, use
22+
:meth:`dpctl.SyclQueue.keep_args_alive`.
23+
24+
Record events that gate the release of objects used by a task with
25+
``add_cleanup_event``, and find them in ``cleanup_events``. They are
26+
waited on, but never become dependencies of later tasks.
27+
28+
Waiting with ``wait`` also drops the references that
29+
:meth:`dpctl.SyclQueue.keep_args_alive` took for tasks that have since
30+
completed, which ``dpctl`` otherwise does on a thread of its own.
31+
32+
.. deprecated:: 0.23.0
33+
``add_event_pair``, ``host_task_events`` and ``num_host_task_events``
34+
are deprecated. Tasks are no longer paired with a host task event, so
35+
``add_event`` takes the computational event alone, and cleanup is
36+
tracked by ``add_cleanup_event``, ``cleanup_events`` and
37+
``num_cleanup_events``.

‎dpctl/CMakeLists.txt‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -198,8 +198,12 @@ endforeach()
198198
set(_cy_file ${CMAKE_CURRENT_SOURCE_DIR}/_sycl_queue.pyx)
199199
get_filename_component(_trgt ${_cy_file} NAME_WLE)
200200
build_dpctl_ext(${_trgt} ${_cy_file} "dpctl" SYCL)
201-
# _sycl_queue include _host_task_util.hpp
202-
target_include_directories(${_trgt} PRIVATE ${CMAKE_CURRENT_SOURCE_DIR})
201+
# _sycl_queue includes _async_dec_ref.hpp, which reaches
202+
# detail/deferred_releases.hpp in the public include directory
203+
target_include_directories(${_trgt} PRIVATE
204+
${CMAKE_CURRENT_SOURCE_DIR}
205+
${CMAKE_CURRENT_SOURCE_DIR}/apis/include
206+
)
203207
target_link_libraries(DpctlCAPI INTERFACE ${_trgt}_headers)
204208

205209
add_subdirectory(compiler)

‎dpctl/_async_dec_ref.hpp‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
//===--- _async_dec_ref.hpp - Implements async DECREF ---------------------===//
2+
//
3+
// Data Parallel Control (dpctl)
4+
//
5+
// Copyright 2026 Intel Corporation
6+
//
7+
// Licensed under the Apache License, Version 2.0 (the "License");
8+
// you may not use this file except in compliance with the License.
9+
// You may obtain a copy of the License at
10+
//
11+
// http://www.apache.org/licenses/LICENSE-2.0
12+
//
13+
// Unless required by applicable law or agreed to in writing, software
14+
// distributed under the License is distributed on an "AS IS" BASIS,
15+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
// See the License for the specific language governing permissions and
17+
// limitations under the License.
18+
//
19+
//===----------------------------------------------------------------------===//
20+
///
21+
/// \file
22+
/// This file implements a utility function to decrement reference counts for a
23+
/// given array of Python objects once a given array of sycl events has
24+
/// completed.
25+
///
26+
/// N.B.: The reference counts are dropped by the thread `dpctl` keeps for
27+
/// running deferred releases, or by a thread that drains them.
28+
///
29+
//===----------------------------------------------------------------------===//
30+
31+
#pragma once
32+
#include <memory>
33+
#include <stddef.h>
34+
#include <stdexcept>
35+
#include <sycl/sycl.hpp>
36+
#include <utility>
37+
#include <vector>
38+
39+
#include "Python.h"
40+
41+
#include "_deferred_release_watcher.hpp"
42+
#include "syclinterface/dpctl_data_types.h"
43+
#include "syclinterface/dpctl_sycl_type_casters.hpp"
44+
45+
namespace
46+
{
47+
48+
/*!
49+
* @brief Copy the events behind `nDepERefs` references into a vector.
50+
*/
51+
std::vector<sycl::event> unwrap_events(DPCTLSyclEventRef *depERefs,
52+
size_t nDepERefs)
53+
{
54+
using dpctl::syclinterface::unwrap;
55+
56+
std::vector<sycl::event> depends;
57+
depends.reserve(nDepERefs);
58+
for (size_t ev_id = 0; ev_id < nDepERefs; ++ev_id) {
59+
depends.push_back(*(unwrap<sycl::event>(depERefs[ev_id])));
60+
}
61+
62+
return depends;
63+
}
64+
65+
/*!
66+
* @brief Schedule DECREFs of `obj_vec` for once `depends` have completed.
67+
*/
68+
void submit_dec_ref(std::vector<PyObject *> obj_vec,
69+
std::vector<sycl::event> depends)
70+
{
71+
dpctl::detail::local_deferred_releases().defer(
72+
std::move(depends), [obj_vec = std::move(obj_vec)]() {
73+
for (PyObject *obj : obj_vec) {
74+
Py_DECREF(obj);
75+
}
76+
});
77+
}
78+
79+
} // namespace
80+
81+
/*!
82+
* @brief Schedule DECREFs of `obj_array` for once `obj_array` is out of use.
83+
*
84+
* An empty kernel is submitted to `QRef` after `depERefs`, and the DECREFs wait
85+
* for it. On an in-order queue submission also orders it after the work already
86+
* submitted there, so the objects are covered regardless of providing dependent
87+
* events. `QRef` should be the queue running the work that uses the objects.
88+
* An event for the kernel is returned.
89+
*
90+
* Sets `*status` to 0 on success and 1 if scheduling threw, and returns nullptr
91+
* on failure. A failed call has scheduled nothing, so the caller still owns the
92+
* references it took.
93+
*/
94+
inline DPCTLSyclEventRef async_dec_ref(DPCTLSyclQueueRef QRef,
95+
PyObject **obj_array,
96+
size_t obj_array_size,
97+
DPCTLSyclEventRef *depERefs,
98+
size_t nDepERefs,
99+
int *status)
100+
{
101+
using dpctl::syclinterface::unwrap;
102+
using dpctl::syclinterface::wrap;
103+
104+
try {
105+
sycl::queue *q = unwrap<sycl::queue>(QRef);
106+
if (!q) {
107+
throw std::invalid_argument("A queue is required");
108+
}
109+
110+
std::vector<sycl::event> depends = unwrap_events(depERefs, nDepERefs);
111+
std::vector<PyObject *> obj_vec(obj_array, obj_array + obj_array_size);
112+
113+
static constexpr int result_ok = 0;
114+
115+
const sycl::event marker =
116+
dpctl::detail::submit_keep_alive_marker(*q, depends);
117+
118+
// allocated before scheduling, as failing afterwards could not be
119+
// reported: the caller would drop a reference the scheduled DECREFs own
120+
std::unique_ptr<sycl::event> e_ptr(new sycl::event(marker));
121+
122+
submit_dec_ref(std::move(obj_vec), {marker});
123+
124+
*status = result_ok;
125+
126+
return wrap<sycl::event>(e_ptr.release());
127+
} catch (...) {
128+
// no exception may escape into the calling Cython code
129+
static constexpr int result_exception = 1;
130+
*status = result_exception;
131+
return nullptr;
132+
}
133+
}

0 commit comments

Comments
 (0)