Skip to content

Commit ea89d1b

Browse files
authored
Merge pull request #2359 from IntelPython/feature/drop-host-task-object-management
Remove `host_task` object management
2 parents 620238f + 5a2d335 commit ea89d1b

23 files changed

Lines changed: 2337 additions & 332 deletions

‎CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77
## [dev] - XXX. XX, XXXX
88

99
### Added
10+
* Added `dpctl.SyclQueue.keep_args_alive` method, which keeps Python objects alive until the work on that queue using them completes [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
11+
* Added `add_event` and `add_cleanup_event` methods and `cleanup_events` and `num_cleanup_events` properties to the order manager, for tracking the events that gate the release of objects used by offloaded tasks [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
12+
* Added `SyclQueue_DeferRelease` C-API function, which defers a release callback until work submitted to a queue completes, and through which `dpctl::utils::keep_args_alive` keeps its arguments alive [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
13+
* Added `dpctl::utils::defer_release` to `dpctl4pybind11.hpp`, which calls a release callback once the given events have completed [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1014

1115
### Changed
16+
* Object lifetime during offload is now managed by a list of releases deferred until the events that gate them have completed, rather than by a `host_task`. `dpctl` keeps one thread for looking over the list and running the releases that have come due, started by the first release deferred and again in the child of a `fork`, which takes the GIL to do so and lets a release call into the interpreter and free USM allocations [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
17+
* `import_dpctl()` now returns 0 on success and -1 with a Python exception set on failure, and `dpctl4pybind11.hpp` raises that exception rather than failing on the first C-API call [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
18+
* The event returned by `dpctl.SyclQueue.keep_args_alive`, `dpctl::utils::keep_args_alive` and `dpctl.SyclQueue._submit_keep_args_alive` is now for an empty kernel that gates the deferred release rather than for a `host_task` that performs it, so its completion means that the objects are no longer in use rather than that they have been released [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1219

1320
### Deprecated
21+
* Deprecated `dpctl.SyclQueue._submit_keep_args_alive` in favor of `dpctl.SyclQueue.keep_args_alive`, which it now calls and which behaves identically [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
22+
* Deprecated the order manager's `add_event_pair`, `host_task_events` and `num_host_task_events`, as `host_task` is no longer used for managing object lifetimes, in favor of `add_event`, `add_cleanup_event`, `cleanup_events` and `num_cleanup_events`; they do not yet emit a `DeprecationWarning` [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1423

1524
### Removed
1625
* Removed the deprecated `dpctl.program` submodule, use `dpctl.compiler` instead [gh-2407](https://github.com/IntelPython/dpctl/pull/2407)
1726

1827
### Fixed
1928
* Removed stale `select_host_device` and `has_host_device` names from `dpctl.__all__`, which made `from dpctl import *` fail with `AttributeError` [gh-2410](https://github.com/IntelPython/dpctl/pull/2410)
29+
* A USM allocation that fails now drops the references held for offloaded tasks that have completed, and is attempted once more if that released anything, so that memory only waiting to be given up is not reported as unavailable [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
2030

2131
### Maintenance
2232
* Removed the `-fno-sycl-id-queries-fit-in-int` compile option, which none of the `dpctl` extensions need since `dpctl.tensor` moved to `dpnp` [gh-2413](https://github.com/IntelPython/dpctl/pull/2413)

‎docs/doc_sources/api_reference/dpctl/utils.rst‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,25 @@
1515

1616
Thread-local object mapping each :class:`dpctl.SyclQueue` to an order
1717
manager, used to ensure sequential ordering of offloaded tasks.
18+
19+
Record submitted tasks with ``add_event`` and use ``submitted_events``
20+
as the dependency list of subsequent submissions. To keep Python objects
21+
referenced by a task alive until it completes, use
22+
:meth:`dpctl.SyclQueue.keep_args_alive`.
23+
24+
Record events that gate the release of objects used by a task with
25+
``add_cleanup_event``, and find them in ``cleanup_events``. They are
26+
waited on, but never become dependencies of later tasks.
27+
28+
Waiting with ``wait`` also drops the references that
29+
:meth:`dpctl.SyclQueue.keep_args_alive` took for tasks that have since
30+
completed, which ``dpctl`` otherwise does on a thread of its own. This is
31+
best-effort: references that thread is already dropping are left to it, so
32+
they may still be held when ``wait`` returns.
33+
34+
.. deprecated:: 0.24.0
35+
``add_event_pair``, ``host_task_events`` and ``num_host_task_events``
36+
are deprecated. Tasks are no longer paired with a host task event, so
37+
``add_event`` takes the computational event alone, and cleanup is
38+
tracked by ``add_cleanup_event``, ``cleanup_events`` and
39+
``num_cleanup_events``.

‎docs/doc_sources/api_reference/dpctl_capi.rst‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,25 @@ API for :c:struct:`PySyclQueueObject`
9797
Note that function does not change the ownership of the ``QRef`` instance and
9898
the caller remains responsible for freeing ``QRef`` as appropriate.
9999
100+
.. c:function:: int SyclQueue_DeferRelease(DPCTLSyclQueueRef QRef, DPCTLSyclEventRef *depERefs, size_t nDepERefs, void (*release)(void *), void *data, DPCTLSyclEventRef markerERef)
101+
102+
:param QRef: queue running the work that uses what ``release`` frees
103+
:param depERefs: array of events for that work
104+
:param nDepERefs: number of events in ``depERefs``
105+
:param release: function to call once the work has completed
106+
:param data: argument passed to ``release``
107+
:param markerERef: existing event the marker event is assigned to
108+
:returns: 0 on success, and 1 if nothing was deferred, in which case
109+
``release`` is never called.
110+
111+
Submits an empty kernel to ``QRef`` after ``depERefs``, assigns its event to
112+
the ``sycl::event`` behind ``markerERef``, and calls ``release(data)`` once
113+
that kernel has completed.
114+
115+
``release`` is called with the GIL held, on a thread owned by :py:mod:`dpctl`
116+
or on one draining the completed releases. Calling the function does not
117+
require the GIL.
118+
100119
API for :c:struct:`PySyclEventObject`
101120
-------------------------------------
102121

‎dpctl/CMakeLists.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,7 @@ endforeach()
198198
set(_cy_file ${CMAKE_CURRENT_SOURCE_DIR}/_sycl_queue.pyx)
199199
get_filename_component(_trgt ${_cy_file} NAME_WLE)
200200
build_dpctl_ext(${_trgt} ${_cy_file} "dpctl" SYCL)
201-
# _sycl_queue include _host_task_util.hpp
201+
# _sycl_queue includes _async_dec_ref.hpp
202202
target_include_directories(${_trgt} PRIVATE ${CMAKE_CURRENT_SOURCE_DIR})
203203
target_link_libraries(DpctlCAPI INTERFACE ${_trgt}_headers)
204204

‎dpctl/_async_dec_ref.hpp‎

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
//===--- _async_dec_ref.hpp - Implements async DECREF ---------------------===//
2+
//
3+
// Data Parallel Control (dpctl)
4+
//
5+
// Copyright 2026 Intel Corporation
6+
//
7+
// Licensed under the Apache License, Version 2.0 (the "License");
8+
// you may not use this file except in compliance with the License.
9+
// You may obtain a copy of the License at
10+
//
11+
// http://www.apache.org/licenses/LICENSE-2.0
12+
//
13+
// Unless required by applicable law or agreed to in writing, software
14+
// distributed under the License is distributed on an "AS IS" BASIS,
15+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
// See the License for the specific language governing permissions and
17+
// limitations under the License.
18+
//
19+
//===----------------------------------------------------------------------===//
20+
///
21+
/// \file
22+
/// This file implements the functions that defer releasing what offloaded work
23+
/// uses until that work has completed: decrementing the reference counts of an
24+
/// array of Python objects, or calling a release function another module
25+
/// passes in.
26+
///
27+
/// N.B.: The reference counts are dropped by the thread `dpctl` keeps for
28+
/// running deferred releases, or by a thread that drains them.
29+
///
30+
//===----------------------------------------------------------------------===//
31+
32+
#pragma once
33+
#include <memory>
34+
#include <stddef.h>
35+
#include <stdexcept>
36+
#include <sycl/sycl.hpp>
37+
#include <utility>
38+
#include <vector>
39+
40+
#include "Python.h"
41+
42+
#include "_deferred_release_watcher.hpp"
43+
#include "syclinterface/dpctl_data_types.h"
44+
#include "syclinterface/dpctl_sycl_type_casters.hpp"
45+
46+
namespace dpctl
47+
{
48+
namespace detail
49+
{
50+
51+
/*!
52+
* @brief Name of the kernel submitted by `submit_keep_alive_marker`.
53+
*/
54+
class keep_alive_marker;
55+
56+
/*!
57+
* @brief An event that gates the release of objects used by work on `q`.
58+
*
59+
* Submits an empty kernel after `deps`. On an in-order queue the submission is
60+
* also ordered after everything already submitted there, so the event covers
61+
* uses of the objects that the caller never tracked.
62+
*/
63+
inline sycl::event
64+
submit_keep_alive_marker(sycl::queue &q, const std::vector<sycl::event> &deps)
65+
{
66+
return q.single_task<keep_alive_marker>(deps, []() {});
67+
}
68+
69+
} // namespace detail
70+
} // namespace dpctl
71+
72+
namespace
73+
{
74+
75+
/*!
76+
* @brief Copy the events behind `nDepERefs` references into a vector.
77+
*/
78+
std::vector<sycl::event> unwrap_events(DPCTLSyclEventRef *depERefs,
79+
size_t nDepERefs)
80+
{
81+
using dpctl::syclinterface::unwrap;
82+
83+
std::vector<sycl::event> depends;
84+
depends.reserve(nDepERefs);
85+
for (size_t ev_id = 0; ev_id < nDepERefs; ++ev_id) {
86+
depends.push_back(*(unwrap<sycl::event>(depERefs[ev_id])));
87+
}
88+
89+
return depends;
90+
}
91+
92+
/*!
93+
* @brief The queue behind `QRef`, which must not be null.
94+
*/
95+
sycl::queue &unwrap_queue(DPCTLSyclQueueRef QRef)
96+
{
97+
sycl::queue *q = dpctl::syclinterface::unwrap<sycl::queue>(QRef);
98+
if (!q) {
99+
throw std::invalid_argument("A queue is required");
100+
}
101+
return *q;
102+
}
103+
104+
/*!
105+
* @brief Schedule DECREFs of `obj_vec` for once `depends` have completed.
106+
*/
107+
void submit_dec_ref(std::vector<PyObject *> obj_vec,
108+
std::vector<sycl::event> depends)
109+
{
110+
dpctl::detail::local_deferred_releases().defer(
111+
std::move(depends), [obj_vec = std::move(obj_vec)]() {
112+
for (PyObject *obj : obj_vec) {
113+
Py_DECREF(obj);
114+
}
115+
});
116+
}
117+
118+
} // namespace
119+
120+
/*!
121+
* @brief Schedule DECREFs of `obj_array` for once `obj_array` is out of use.
122+
*
123+
* An empty kernel is submitted to `QRef` after `depERefs`, and the DECREFs wait
124+
* for it. On an in-order queue submission also orders it after the work already
125+
* submitted there, so the objects are covered regardless of providing dependent
126+
* events. `QRef` should be the queue running the work that uses the objects.
127+
* An event for the kernel is returned.
128+
*
129+
* Sets `*status` to 0 on success and 1 if scheduling threw, and returns nullptr
130+
* on failure. A failed call has scheduled nothing, so the caller still owns the
131+
* references it took.
132+
*/
133+
inline DPCTLSyclEventRef async_dec_ref(DPCTLSyclQueueRef QRef,
134+
PyObject **obj_array,
135+
size_t obj_array_size,
136+
DPCTLSyclEventRef *depERefs,
137+
size_t nDepERefs,
138+
int *status)
139+
{
140+
using dpctl::syclinterface::wrap;
141+
142+
try {
143+
sycl::queue &q = unwrap_queue(QRef);
144+
145+
std::vector<sycl::event> depends = unwrap_events(depERefs, nDepERefs);
146+
std::vector<PyObject *> obj_vec(obj_array, obj_array + obj_array_size);
147+
148+
static constexpr int result_ok = 0;
149+
150+
const sycl::event marker =
151+
dpctl::detail::submit_keep_alive_marker(q, depends);
152+
153+
// allocated before scheduling, as failing afterwards could not be
154+
// reported: the caller would drop a reference the scheduled DECREFs own
155+
std::unique_ptr<sycl::event> e_ptr(new sycl::event(marker));
156+
157+
submit_dec_ref(std::move(obj_vec), {marker});
158+
159+
*status = result_ok;
160+
161+
return wrap<sycl::event>(e_ptr.release());
162+
} catch (...) {
163+
// no exception may escape into the calling Cython code
164+
static constexpr int result_exception = 1;
165+
*status = result_exception;
166+
return nullptr;
167+
}
168+
}
169+
170+
/*!
171+
* @brief Call `release(data)` once the work using what it releases is done.
172+
*
173+
* Submits an empty kernel to `QRef` after `depERefs`, as `async_dec_ref` does,
174+
* and assigns its event to the `sycl::event` behind `markerERef`. `release` is
175+
* called with the GIL held, on `dpctl`'s thread or on one that drains.
176+
*
177+
* @return 0 on success, and 1 if nothing was deferred, in which case `release`
178+
* is never called.
179+
*/
180+
inline int defer_release(DPCTLSyclQueueRef QRef,
181+
DPCTLSyclEventRef *depERefs,
182+
size_t nDepERefs,
183+
void (*release)(void *),
184+
void *data,
185+
DPCTLSyclEventRef markerERef)
186+
{
187+
using dpctl::syclinterface::unwrap;
188+
189+
try {
190+
sycl::queue &q = unwrap_queue(QRef);
191+
sycl::event *marker = unwrap<sycl::event>(markerERef);
192+
if (!release || !marker) {
193+
throw std::invalid_argument("A release and an event are required");
194+
}
195+
196+
*marker = dpctl::detail::submit_keep_alive_marker(
197+
q, unwrap_events(depERefs, nDepERefs));
198+
199+
dpctl::detail::local_deferred_releases().defer(
200+
{*marker}, [release, data]() { release(data); });
201+
202+
return 0;
203+
} catch (...) {
204+
// no exception may escape into the calling Cython code
205+
return 1;
206+
}
207+
}

0 commit comments

Comments
 (0)