Skip to content

Commit 2ed816f

Browse files
authored
Merge pull request #2299 from IntelPython/no-op-order-manager-for-in-order-queues
Use a no-op order manager for in-order queues
2 parents 3c6b494 + c2c6c2b commit 2ed816f

10 files changed

Lines changed: 492 additions & 5 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1111
* Added `add_event` and `add_cleanup_event` methods and `cleanup_events` and `num_cleanup_events` properties to the order manager, for tracking the events that gate the release of objects used by offloaded tasks [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1212
* Added `SyclQueue_DeferRelease` C-API function, which defers a release callback until work submitted to a queue completes, and through which `dpctl::utils::keep_args_alive` keeps its arguments alive [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1313
* Added `dpctl::utils::defer_release` to `dpctl4pybind11.hpp`, which calls a release callback once the given events have completed [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
14+
* Added `dpctl.SyclQueue.get_last_event` and `dpctl.SyclQueue.is_get_last_event_available`, and the `DPCTLQueue_GetLastEvent` and `DPCTLQueue_GetLastEvent_Available` C-API functions, wrapping the DPC++ `sycl_ext_oneapi_in_order_queue_events` extension [gh-2299](https://github.com/IntelPython/dpctl/pull/2299)
1415

1516
### Changed
1617
* Object lifetime during offload is now managed by a list of releases deferred until the events that gate them have completed, rather than by a `host_task`. `dpctl` keeps one thread for looking over the list and running the releases that have come due, started by the first release deferred and again in the child of a `fork`, which takes the GIL to do so and lets a release call into the interpreter and free USM allocations [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1718
* `import_dpctl()` now returns 0 on success and -1 with a Python exception set on failure, and `dpctl4pybind11.hpp` raises that exception rather than failing on the first C-API call [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
1819
* The event returned by `dpctl.SyclQueue.keep_args_alive`, `dpctl::utils::keep_args_alive` and `dpctl.SyclQueue._submit_keep_args_alive` is now for an empty kernel that gates the deferred release rather than for a `host_task` that performs it, so its completion means that the objects are no longer in use rather than that they have been released [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)
20+
* `dpctl.utils.SequentialOrderManager[q].submitted_events` includes the queue's last event for in-order queues when `sycl_ext_oneapi_in_order_queue_events` is available, and `wait()` waits for all work on an in-order queue [gh-2299](https://github.com/IntelPython/dpctl/pull/2299)
1921

2022
### Deprecated
2123
* Deprecated `dpctl.SyclQueue._submit_keep_args_alive` in favor of `dpctl.SyclQueue.keep_args_alive`, which it now calls and which behaves identically [gh-2359](https://github.com/IntelPython/dpctl/pull/2359)

‎docs/doc_sources/api_reference/dpctl/utils.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,15 @@
3131
best-effort: references that thread is already dropping are left to it, so
3232
they may still be held when ``wait`` returns.
3333

34+
For an in-order queue, if
35+
:meth:`dpctl.SyclQueue.is_get_last_event_available` is ``True``,
36+
``submitted_events`` also has the event of the last command submitted to
37+
the queue. That covers tasks submitted to the queue without being added
38+
to the order manager, e.g. by other threads, and may be the event of an
39+
empty kernel submitted by :meth:`dpctl.SyclQueue.keep_args_alive`.
40+
``wait``, ``clear`` and thread exit wait for all work on the queue,
41+
including work submitted by other threads.
42+
3443
.. deprecated:: 0.24.0
3544
``add_event_pair``, ``host_task_events`` and ``num_host_task_events``
3645
are deprecated. Tasks are no longer paired with a host task event, so

‎dpctl/_backend.pxd‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -756,6 +756,8 @@ cdef extern from "syclinterface/dpctl_sycl_queue_interface.h":
756756
size_t Count,
757757
int Advice)
758758
cdef bool DPCTLQueue_IsInOrder(const DPCTLSyclQueueRef QRef)
759+
cdef bool DPCTLQueue_GetLastEvent_Available()
760+
cdef DPCTLSyclEventRef DPCTLQueue_GetLastEvent(const DPCTLSyclQueueRef QRef)
759761
cdef DPCTLSyclEventRef DPCTLQueue_SubmitBarrier(
760762
const DPCTLSyclQueueRef QRef)
761763
cdef DPCTLSyclEventRef DPCTLQueue_SubmitBarrierForEvents(

‎dpctl/_sycl_queue.pyx‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,8 @@ from ._backend cimport ( # noqa: E211
5353
DPCTLQueue_GetBackend,
5454
DPCTLQueue_GetContext,
5555
DPCTLQueue_GetDevice,
56+
DPCTLQueue_GetLastEvent,
57+
DPCTLQueue_GetLastEvent_Available,
5658
DPCTLQueue_HasEnableProfiling,
5759
DPCTLQueue_Hash,
5860
DPCTLQueue_IsInOrder,
@@ -2330,6 +2332,48 @@ cdef class SyclQueue(_SyclQueue):
23302332

23312333
return SyclEvent._create(ERef)
23322334

2335+
@staticmethod
2336+
def is_get_last_event_available():
2337+
"""Returns ``True`` if dpctl was built with a compiler that supports
2338+
the DPC++ ``sycl_ext_oneapi_in_order_queue_events`` extension used by
2339+
:meth:`dpctl.SyclQueue.get_last_event`.
2340+
"""
2341+
return DPCTLQueue_GetLastEvent_Available()
2342+
2343+
def get_last_event(self):
2344+
"""
2345+
get_last_event()
2346+
2347+
Returns the event of the last command submitted to an in-order queue.
2348+
2349+
Wraps ``sycl::queue::ext_oneapi_get_last_event()`` from the
2350+
experimental DPC++ ``sycl_ext_oneapi_in_order_queue_events``
2351+
extension.
2352+
2353+
Returns:
2354+
dpctl.SyclEvent | None:
2355+
Event of the last submitted command, or ``None`` if no
2356+
command was submitted to the queue.
2357+
2358+
Raises:
2359+
RuntimeError:
2360+
If the extension is not available, see
2361+
:meth:`dpctl.SyclQueue.is_get_last_event_available`.
2362+
ValueError:
2363+
If the queue is not in-order.
2364+
"""
2365+
cdef DPCTLSyclEventRef ERef = NULL
2366+
if not DPCTLQueue_GetLastEvent_Available():
2367+
raise RuntimeError(
2368+
"sycl_ext_oneapi_in_order_queue_events extension not available"
2369+
)
2370+
if not DPCTLQueue_IsInOrder(self._queue_ref):
2371+
raise ValueError("get_last_event requires an in-order queue")
2372+
ERef = DPCTLQueue_GetLastEvent(self._queue_ref)
2373+
if ERef is NULL:
2374+
return None
2375+
return SyclEvent._create(ERef)
2376+
23332377
@property
23342378
def name(self):
23352379
"""Returns the device name for the device

‎dpctl/tests/test_sycl_queue.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,36 @@ def test_queue_submit_barrier(valid_filter):
169169
q.submit_barrier(range(3))
170170

171171

172+
def test_queue_is_get_last_event_available():
173+
assert isinstance(dpctl.SyclQueue.is_get_last_event_available(), bool)
174+
175+
176+
def test_queue_get_last_event():
177+
if not dpctl.SyclQueue.is_get_last_event_available():
178+
pytest.skip("sycl_ext_oneapi_in_order_queue_events not available")
179+
try:
180+
q = dpctl.SyclQueue(property="in_order")
181+
except dpctl.SyclQueueCreationError:
182+
pytest.skip("Queue could not be created for default-selected device")
183+
buf = dpctl.memory.MemoryUSMDevice(1024, queue=q)
184+
ev = q.memset_async(buf, 1, 1024)
185+
last_ev = q.get_last_event()
186+
assert isinstance(last_ev, dpctl.SyclEvent)
187+
last_ev.wait()
188+
assert ev.execution_status == dpctl.event_status_type.complete
189+
190+
191+
def test_queue_get_last_event_out_of_order():
192+
if not dpctl.SyclQueue.is_get_last_event_available():
193+
pytest.skip("sycl_ext_oneapi_in_order_queue_events not available")
194+
try:
195+
q = dpctl.SyclQueue()
196+
except dpctl.SyclQueueCreationError:
197+
pytest.skip("Queue could not be created for default-selected device")
198+
with pytest.raises(ValueError):
199+
q.get_last_event()
200+
201+
172202
def test_queue__repr__():
173203
try:
174204
q1 = dpctl.SyclQueue(property=0)

‎dpctl/tests/test_utils.py‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
import time
2525
import warnings
2626

27+
import numpy as np
2728
import pytest
2829

2930
import dpctl
@@ -34,6 +35,7 @@
3435
_start_deferred_release_watcher,
3536
_stop_deferred_release_watcher,
3637
)
38+
from dpctl.utils import _order_manager
3739

3840
_free_threaded = bool(sysconfig.get_config_var("Py_GIL_DISABLED"))
3941

@@ -503,3 +505,168 @@ def __del__(self):
503505
timeout=120,
504506
)
505507
assert out == "4"
508+
509+
510+
def _in_order_queue(*args, cls=dpctl.SyclQueue):
511+
try:
512+
return cls(*args, property="in_order")
513+
except dpctl.SyclQueueCreationError:
514+
pytest.skip("Queue could not be created for default-selected device")
515+
516+
517+
@pytest.fixture(params=[True, False], ids=["last_event", "added_events"])
518+
def use_last_event(request, monkeypatch):
519+
if request.param and not dpctl.SyclQueue.is_get_last_event_available():
520+
pytest.skip("sycl_ext_oneapi_in_order_queue_events not available")
521+
monkeypatch.setattr(_order_manager, "_use_last_event", request.param)
522+
return request.param
523+
524+
525+
def test_in_order_order_manager():
526+
q = _in_order_queue()
527+
_som = dpctl.utils.SequentialOrderManager
528+
_mngr = _som[q]
529+
assert _som[q] is _mngr
530+
assert isinstance(_mngr, _order_manager._InOrderManager)
531+
assert isinstance(_mngr.submitted_events, list)
532+
assert isinstance(_mngr.cleanup_events, list)
533+
_mngr.add_event(dpctl.SyclEvent())
534+
_mngr.add_event([dpctl.SyclEvent(), dpctl.SyclEvent()])
535+
_mngr.add_cleanup_event(dpctl.SyclEvent())
536+
_mngr.add_cleanup_event([dpctl.SyclEvent(), dpctl.SyclEvent()])
537+
# complete events are not kept
538+
assert _mngr.num_cleanup_events == 0
539+
_mngr.wait()
540+
cpy = _mngr.__copy__()
541+
assert isinstance(cpy, _order_manager._InOrderManager)
542+
assert cpy.num_cleanup_events == 0
543+
_som.clear()
544+
del cpy
545+
546+
547+
@pytest.mark.parametrize("in_order", [True, False])
548+
def test_order_manager_add_non_event(in_order):
549+
q = _in_order_queue() if in_order else dpctl.SyclQueue()
550+
_mngr = dpctl.utils.SequentialOrderManager[q]
551+
with pytest.raises(TypeError):
552+
_mngr.add_event(None)
553+
with pytest.raises(TypeError):
554+
_mngr.add_cleanup_event([None])
555+
dpctl.utils.SequentialOrderManager.clear()
556+
557+
558+
def test_in_order_order_manager_host_task_api():
559+
q = _in_order_queue()
560+
_som = dpctl.utils.SequentialOrderManager
561+
_mngr = _som[q]
562+
with warnings.catch_warnings():
563+
warnings.simplefilter("error")
564+
_mngr.add_event_pair(dpctl.SyclEvent(), dpctl.SyclEvent())
565+
_mngr.add_event_pair([dpctl.SyclEvent()], dpctl.SyclEvent())
566+
assert _mngr.host_task_events == []
567+
assert _mngr.num_host_task_events == 0
568+
_som.clear()
569+
570+
571+
def test_in_order_order_manager_wait_drops_references(stopped_release_watcher):
572+
q = _in_order_queue()
573+
_som = dpctl.utils.SequentialOrderManager
574+
_mngr = _som[q]
575+
576+
released = []
577+
args = (_Sentinel(released),)
578+
ev = q.keep_args_alive(args)
579+
del args
580+
ev.wait()
581+
assert not released, "dpctl's thread is stopped, so nothing has run it"
582+
583+
_mngr.wait()
584+
assert released == [True]
585+
_som.clear()
586+
587+
588+
def test_in_order_order_manager_submitted_events(use_last_event):
589+
q = _in_order_queue()
590+
_mngr = dpctl.utils.SequentialOrderManager[q]
591+
buf = dpctl.memory.MemoryUSMDevice(1024, queue=q)
592+
# work which is not added to the order manager
593+
q.memset_async(buf, 1, 1024)
594+
if use_last_event:
595+
assert _mngr.num_submitted_events == 1
596+
else:
597+
assert _mngr.num_submitted_events == 0
598+
dpctl.utils.SequentialOrderManager.clear()
599+
600+
601+
def _fill_in_steps(q, buf, n_fills, mngr=None):
602+
nbytes = buf.nbytes
603+
ev = None
604+
for v in range(1, n_fills + 1):
605+
ev = q.memset_async(buf, v, nbytes, dEvents=[ev] if ev else None)
606+
if mngr is not None:
607+
mngr.add_event(ev)
608+
mngr.add_cleanup_event(q.keep_args_alive((buf,), [ev]))
609+
return ev
610+
611+
612+
def test_in_order_order_manager_other_queue_depends():
613+
if not dpctl.SyclQueue.is_get_last_event_available():
614+
pytest.skip("sycl_ext_oneapi_in_order_queue_events not available")
615+
q_in = _in_order_queue()
616+
q_other = dpctl.SyclQueue(q_in.sycl_context, q_in.sycl_device)
617+
_mngr = dpctl.utils.SequentialOrderManager[q_in]
618+
n_fills = 8
619+
buf = dpctl.memory.MemoryUSMDevice(64 * 1024 * 1024, queue=q_in)
620+
host = np.empty(buf.nbytes, dtype=np.uint8)
621+
for _ in range(5):
622+
q_in.memset(buf, 0, buf.nbytes)
623+
_fill_in_steps(q_in, buf, n_fills - 1, _mngr)
624+
# the last fill is not added to the order manager
625+
q_in.memset_async(buf, n_fills, buf.nbytes)
626+
q_other.memcpy_async(
627+
host, buf, buf.nbytes, dEvents=_mngr.submitted_events
628+
).wait()
629+
assert np.all(host == n_fills)
630+
dpctl.utils.SequentialOrderManager.clear()
631+
632+
633+
def test_in_order_order_manager_other_queue_event(use_last_event):
634+
q_in = _in_order_queue()
635+
q_other = dpctl.SyclQueue(q_in.sycl_context, q_in.sycl_device)
636+
_mngr = dpctl.utils.SequentialOrderManager[q_in]
637+
n_fills = 8
638+
buf = dpctl.memory.MemoryUSMDevice(64 * 1024 * 1024, queue=q_in)
639+
host = np.empty(buf.nbytes, dtype=np.uint8)
640+
for _ in range(5):
641+
q_in.memset(buf, 0, buf.nbytes)
642+
_mngr.add_event(_fill_in_steps(q_other, buf, n_fills))
643+
q_in.memcpy_async(
644+
host, buf, buf.nbytes, dEvents=_mngr.submitted_events
645+
).wait()
646+
assert np.all(host == n_fills)
647+
q_other.wait()
648+
dpctl.utils.SequentialOrderManager.clear()
649+
650+
651+
class _WaitCountingQueue(dpctl.SyclQueue):
652+
def __init__(self, *args, **kwargs):
653+
self.num_waits = 0
654+
655+
def wait(self):
656+
self.num_waits += 1
657+
super().wait()
658+
659+
660+
def test_in_order_order_manager_waits_on_queue():
661+
q = _in_order_queue(cls=_WaitCountingQueue)
662+
_som = dpctl.utils.SequentialOrderManager
663+
_mngr = _som[q]
664+
_mngr.wait()
665+
assert q.num_waits == 1
666+
cpy = _mngr.__copy__()
667+
del cpy
668+
assert q.num_waits == 2
669+
del _mngr
670+
# clear() runs at exit, and must wait on in-order queues as well
671+
_som.clear()
672+
assert q.num_waits > 2

0 commit comments

Comments
 (0)