Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ This release drops support for Python 3.10, making Python 3.11 the minimum requi
* Added the multivariate distributions, `permutation`, `shuffle`, `permuted`, `choice` and `bytes` to `dpnp.random.Generator` [#3160](https://github.com/IntelPython/dpnp/pull/3160)
* Added the `dpnp.random.PCG64DXSM` bit generator, which matches `numpy.random.PCG64DXSM` for raw values, `random`, `uniform` and `bytes` [#3161](https://github.com/IntelPython/dpnp/pull/3161)
* Added the `dpnp.random.MRG32k3a` bit generator [#3161](https://github.com/IntelPython/dpnp/pull/3161)
* Added `dpnp.random.get_state`, `dpnp.random.set_state`, `dpnp.random.get_bit_generator` and `dpnp.random.set_bit_generator`, and support for a bit generator as the seed of `dpnp.random.RandomState` [#3162](https://github.com/IntelPython/dpnp/pull/3162)

### Changed

Expand All @@ -32,6 +33,7 @@ This release drops support for Python 3.10, making Python 3.11 the minimum requi
* Reduced the overhead of `dpnp.ndarray.flags` and `dpnp.tensor.usm_ndarray.flags` [#3168](https://github.com/IntelPython/dpnp/pull/3168)
* Changed `dpnp.linalg.LinAlgError` to derive from `numpy.linalg.LinAlgError`, so `except numpy.linalg.LinAlgError` also catches errors raised by dpnp linear algebra functions [#3184](https://github.com/IntelPython/dpnp/pull/3184)
* Bumped the minimum required `dpctl` version to `0.24.0` [#3169](https://github.com/IntelPython/dpnp/pull/3169)
* Reimplemented `dpnp.random.RandomState` and the `dpnp.random` functions on device kernels, drawing on the queue of the generator. `device`, `usm_type` and `sycl_queue` are now keyword-only, `usm_type` defaults to the USM type of the array parameters, `randint(dtype=int)` returns `int64` and `shuffle` requires a `dpnp.ndarray` or `usm_ndarray` [#3162](https://github.com/IntelPython/dpnp/pull/3162)

### Deprecated

Expand All @@ -53,6 +55,8 @@ This release drops support for Python 3.10, making Python 3.11 the minimum requi
* Fixed `dpnp.place` and `dpnp.tensor.place` to order the write after the cast of `vals` to the array's data type, which could put wrong values into the array on an out-of-order queue [#3164](https://github.com/IntelPython/dpnp/pull/3164)
* Fixed the example of handling duplicate indices in the `dpnp.tensor.put` docstring, which imported `dpctl.tensor`, did not parse and lost writes when `x` is longer than `indices` [#3165](https://github.com/IntelPython/dpnp/pull/3165)
* Fixed a segfault when setting `writable` on a `Flags` object created without an array [#3168](https://github.com/IntelPython/dpnp/pull/3168)
* Fixed `geometric`, `pareto` and `laplace` of `dpnp.random` and `dpnp.random.RandomState` to give NumPy's distributions: `geometric` counted the failures instead of the trials, `pareto` drew from the Pareto I distribution instead of the Lomax one, and `laplace` with `scale=0` returned 0 instead of `loc` [#3162](https://github.com/IntelPython/dpnp/pull/3162)
Comment thread
ndgrigorian marked this conversation as resolved.
Resolved
* Fixed `multivariate_normal`, `multinomial` and `shuffle` of `dpnp.random` and `dpnp.random.RandomState`, which copied every input to a new queue, so that `multivariate_normal` raised `ValueError` and each call of the others left a queue alive [#3162](https://github.com/IntelPython/dpnp/pull/3162)

### Security

Expand Down
58 changes: 36 additions & 22 deletions doc/reference/random.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ DPNP draws random numbers on the SYCL device, by default with the
counter-based Philox4x32x10 generator of the oneMKL device RNG API, or with
NumPy's PCG64DXSM generator or the MRG32k3a generator of the oneMKL device
RNG API. A oneMath build takes Philox4x32x10 and MRG32k3a from oneMath's
device RNG API instead. It provides the :class:`Generator` API of NumPy,
recommended for new code. The legacy :class:`RandomState` API and the
module-level functions still use the host-side oneMKL RNG.
device RNG API instead. It provides both NumPy random APIs: the
:class:`Generator` API recommended for new code, and the legacy
:class:`RandomState` API with the module-level functions built on it.

.. code-block:: python

Expand All @@ -27,6 +27,8 @@ A generator draws all its samples on one SYCL queue, chosen with the
``device`` or ``sycl_queue`` keyword when it is created. All samples are
:class:`dpnp.ndarray`, zero-dimensional when ``size`` is ``None``.

.. _random-differences:

Differences from NumPy
----------------------

Expand All @@ -52,8 +54,8 @@ Differences from NumPy
default, the samples take the USM type of the array parameters, or
``"device"`` when there are none.
* Parameters on the host (Python scalars, lists and NumPy arrays) are
checked as in NumPy. Parameters in :class:`dpnp.ndarray` are
not, since that would wait for the device.
checked as in NumPy. In :class:`Generator`, parameters in
:class:`dpnp.ndarray` are not, since that would wait for the device.
Invalid ones give undefined values, such as NaN or the smallest ``int64``,
but the kernels handle them safely: they never loop without end or access
memory out of bounds. Floating-point arrays given for integer parameters
Expand All @@ -72,15 +74,21 @@ Differences from NumPy
* ``zipf`` follows the Zipf distribution truncated to the largest ``int64``.
NumPy's rejection test rounds to accepting every sample above ``2**53``,
which makes its tail too heavy for ``a`` close to 1.
* ``hypergeometric`` requires ``ngood`` and ``nbad`` below ``10**9``, and
``multivariate_hypergeometric`` requires ``sum(colors)`` below ``10**9``
for the ``"count"`` method too.
* :meth:`RandomState.randint` and :func:`randint` return ``int64`` values by
default, as do :meth:`RandomState.random_integers`,
:func:`random_integers` and :meth:`RandomState.tomaxint`, whose values
reach the largest ``int64``. NumPy returns a C ``long`` instead, which is
``int32`` on Windows.
* ``hypergeometric`` requires ``ngood`` and ``nbad`` below ``10**9``, also in
:class:`RandomState`, and ``multivariate_hypergeometric`` requires
``sum(colors)`` below ``10**9`` for the ``"count"`` method too.
* ``choice`` always returns a sample without replacement in random order,
so its ``shuffle`` parameter has no effect.
* ``p`` of ``choice`` and ``pvals`` of ``multinomial`` are only checked when
they are on the host, as the other parameters are.
* :meth:`Generator.shuffle` only accepts an array on the device, a
:class:`dpnp.ndarray` or :class:`dpnp.tensor.usm_ndarray`, as in CuPy.
* :meth:`Generator.shuffle` and :func:`shuffle` only accept an array on the
device, a :class:`dpnp.ndarray` or :class:`dpnp.tensor.usm_ndarray`, as in
CuPy.
* Generators and bit generators cannot be pickled or deep-copied,
and :class:`Philox4x3210` and :class:`MRG32k3a` have no ``advance`` or
``jumped`` method.
Expand Down Expand Up @@ -201,6 +209,24 @@ Legacy random generation

.. hint:: `NumPy API Reference: Legacy random generation <https://numpy.org/doc/stable/reference/random/legacy.html>`_

:class:`RandomState` keeps the parameter checks of NumPy's legacy one, also
for parameters in :class:`dpnp.ndarray`, as CuPy's does, so that a call with
such parameters waits for the device to check them. The module-level
functions use one :class:`RandomState` per SYCL queue: the
queue given by the ``device`` or ``sycl_queue`` keyword, or else the queue of
a :class:`dpnp.ndarray` parameter, or else the default queue.

.. autosummary::
:toctree: generated/
:nosignatures:

RandomState
seed
get_state
set_state
get_bit_generator
set_bit_generator

Simple random data
^^^^^^^^^^^^^^^^^^

Expand Down Expand Up @@ -271,15 +297,3 @@ Distributions
wald
weibull
zipf

RandomState
^^^^^^^^^^^

.. autosummary::
:toctree: generated/
:nosignatures:

RandomState
seed
get_random_state
set_random_state
4 changes: 2 additions & 2 deletions dpnp/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
- `dpnp_algo/dpnp_elementwise_common.py`: ufunc wrapper classes (`DPNPUnaryFunc`, `DPNPBinaryFunc`, two-output and specialised variants)
- `dpnp_utils/`: shared helpers (`get_usm_allocations`, `map_dtype_to_device`, reduction wrappers)
- `linalg/`, `fft/`, `random/`, `scipy/`, `lib/`: subpackages, each exporting through its own `__init__.py`
- `random/`: `_bit_generator.py` (`BitGenerator`, `Philox4x3210`, `PCG64DXSM`, `MRG32k3a`; `SeedSequence` is NumPy's) and `_generator.py` (`Generator`, which implements every distribution). The kernels are in `dpnp.libtensor._rng_impl`.
- `random/`: `_bit_generator.py` (`BitGenerator`, `Philox4x3210`, `PCG64DXSM`, `MRG32k3a`; `SeedSequence` is NumPy's), `_generator.py` (`Generator`, which implements every distribution), `dpnp_random_state.py` (`RandomState`, the same samplers with NumPy's legacy checks) and `dpnp_iface_random.py` (module functions, one `RandomState` per queue). The kernels are in `dpnp.libtensor._rng_impl`.
- Legacy, don't extend: the Cython `dpnp_algo` path and `dpnp_backend_c` (still used by `dpnp.partition` and `dpnp.random`), and the NumPy fallback `call_origin`. New kernels go into `dpnp/libtensor` (see `dpnp/libtensor/AGENTS.md`).

## Adding a Public Function
Expand All @@ -21,7 +21,7 @@
- Extension calls are ordered with `_manager = dpctl.utils.SequentialOrderManager[exec_q]`: pass `depends=_manager.submitted_events`, then `_manager.add_event_pair(ht_ev, ev)`. Don't add blocking waits.
- Use `dpnp.default_float_type(...)` or `map_dtype_to_device(...)` for the default float type.
- Deprecated ufuncs use `DPNPDeprecatedUnaryFunc(..., deprecated_msg=...)`.
- A `dpnp.random` distribution is a `Generator` method: it calls `self._check_queues(...)` on its array parameters (raises `ExecutionPlacementError`), checks host parameters as NumPy does, and draws through `_cont`, `_float_sample` or `_discrete_sample` into `BitGenerator._fill`. Parameters on the device are not checked, as the check waits for the device. With `usm_type=None`, the output and the temporaries take the coerced USM type of the array parameters (`Generator._sample`).
- A `dpnp.random` distribution is a `Generator` method: it calls `self._check_queues(...)` on its array parameters (raises `ExecutionPlacementError`), checks host parameters as NumPy does, and draws through `_cont`, `_float_sample` or `_discrete_sample` into `BitGenerator._fill`. `RandomState` and the module function delegate to it. Parameters on the device are checked only by `RandomState` (`_check_device`), as the check waits for the device. With `usm_type=None`, the output and the temporaries take the coerced USM type of the array parameters (`Generator._sample`).
- Unsupported NumPy keywords raise `NotImplementedError` (`dpnp.check_limitations`) and are listed under `Limitations` in the docstring.
- The Array API spec version is set in three places that change together: `__array_api_version__` in `dpnp/tensor/_array_api.py`, `ARRAY_API_TESTS_VERSION` in `.github/workflows/conda-package.yml` and `doc/reference/array_api.rst` (plus the version test in `dpnp/tests/test_ndarray.py`). Inspection utilities come from `__array_namespace_info__` in `dpnp_array_api_info.py`.
- `dpnp.tensor` replaces `dpctl.tensor`, which no longer exists here; `SyclQueue`, `SyclDevice`, `dpctl.memory` and `SequentialOrderManager` still come from `dpctl`.
Expand Down
8 changes: 8 additions & 0 deletions dpnp/random/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@
f,
gamma,
geometric,
get_bit_generator,
get_state,
gumbel,
hypergeometric,
laplace,
Expand All @@ -71,6 +73,8 @@
rayleigh,
sample,
seed,
set_bit_generator,
set_state,
shuffle,
standard_cauchy,
standard_exponential,
Expand Down Expand Up @@ -104,6 +108,8 @@
"f",
"gamma",
"geometric",
"get_bit_generator",
"get_state",
"gumbel",
"hypergeometric",
"laplace",
Expand Down Expand Up @@ -131,6 +137,8 @@
"sample",
"shuffle",
"seed",
"set_bit_generator",
"set_state",
"standard_cauchy",
"standard_exponential",
"standard_gamma",
Expand Down
Loading
Loading