Skip to content

Commit d2b5374

Browse files
authored
Add cho_factor and cho_solve to dpnp.scipy.linalg (#3116)
Add SciPy-compatible `dpnp.scipy.linalg.cho_factor` and `dpnp.scipy.linalg.cho_solve`, including batched input as in SciPy 1.18 (#3096). ### Implementation - `cho_factor` uses the existing `potrf` and `potrf_batch` bindings. Only the referenced triangle is copied, in one pass, into a zeroed C-order buffer, which matches the SciPy 1.18 output. With `overwrite_a=True`, a 2-D, writable input of the result dtype that is C- or F-contiguous is factored in place. As in SciPy, a 0-D or 1-D input is viewed as a matrix. - `cho_solve` uses a new LAPACK `potrs` binding for a 2-D factor. For a batch of factors it calls a new BLAS `trsm_batch` binding twice, since oneMKL `potrs_batch` is much slower (MKLD-20341). - A single factor (2-D, or with size-1 batch dimensions) is not broadcast to a batched right-hand side: the batch is folded into the columns of one `potrs` call. For a real type on GPU with more than 16 folded columns and a batch of at most 32, the factor is shared by the batch in `trsm_batch` (stride 0) instead, because double-precision `trsm` on Xe GPUs slows down by up to 3.6x above 16 columns. - Batch dimensions broadcast as in `lu_solve`, and a broadcast right-hand side is copied without being materialized first. - `lower` can also be the batch-shaped array SciPy 1.18 returns, or a sequence, if all its values are equal. An empty `lower` raises `ValueError`, as in SciPy, except in `cho_solve` for a zero-size batch of factors. - With `overwrite_b=True`, `cho_solve` solves in a copy when `b` overlaps the factor (SciPy overwrites the factor). - With `check_finite=True`, both finiteness checks of `cho_solve` are submitted before the first host sync. ### Differences from SciPy Documented in the docstrings and covered by tests: - `cho_factor` returns `lower` as a Python bool; SciPy 1.18 returns an array tiled to the batch shape. - `cho_solve` supports a zero-size batch; SciPy raises. - `lower` with unequal values raises `ValueError` in `cho_factor` and `cho_solve`; SciPy 1.18 accepts it. - `cho_solve` with `overwrite_b=True` and `b` overlapping the factor leaves the factor unchanged; SciPy overwrites it. As in the existing `lu_factor` and `dpnp.linalg.cholesky`, integer and boolean inputs are cast to the default floating type of the device; SciPy 1.18 uses float32 for boolean and up to 16-bit integer inputs. ### Other changes - `dpnp.linalg.cholesky`: a See Also link to `dpnp.scipy.linalg.cho_factor` and a fixed example output for `upper=True`. - `doc/reference/linalg.rst`: removed the stale `linalg.lu_factor` and `linalg.lu_solve` entries. - Backported `tests/cupyx_tests/linalg_tests/test_solve.py` from CuPy, with `TestChosolve` adapted to `dpnp.scipy.linalg.cho_solve` and the `invh` tests skipped. ### Testing - New tests in `test_linalg.py`, `test_sycl_queue.py` and `test_usm_type.py`, plus the CuPy tests. - On Panther Lake H (Level Zero GPU, OpenCL GPU and OpenCL CPU devices), the linalg, SYCL queue and USM type tests pass, and a comparison with SciPy 1.18.1 (6720 checks over dtypes including mixed ones, C and F layouts, `lower`, batching and broadcasting, `overwrite_*` and `check_finite`) finds no differences. Earlier revisions were also tested on PVC, Lunar Lake and Arc B580. - The build adds no compiler warnings. ### Performance `cho_factor` + `cho_solve` against `dpnp.linalg.solve`, median times, measured on an earlier revision of this branch: | Device | n=4096 | batch (1000, 32, 32) | re-solve: `cho_solve` vs `lu_solve` | | --- | --- | --- | --- | | PVC Max 1100 GPU | 3.1x faster (f64) | 84x faster (f64) | 1.0–2.0x faster | | Panther Lake H Xe3 GPU | 1.8–2.3x faster | 10–21x faster | 1.2–4.5x faster | | Arc B580 | 1.7–2.2x faster | 20–43x faster | 1.07–3.2x faster | On client and Battlemage GPUs, the Cholesky path is slower than LU for small single matrices (n=256: 0.5–0.6x) and for batches of larger matrices such as (100, 128, 128) (0.24–0.5x), because of oneMKL issues in `potrf` and `potrf_batch`.
1 parent 56ca787 commit d2b5374

22 files changed

Lines changed: 2348 additions & 7 deletions

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ This release drops support for Python 3.10, making Python 3.11 the minimum requi
1313
* Added support for boolean input in `dpnp.positive` and the `+` operator of `dpnp.ndarray` [#3128](https://github.com/IntelPython/dpnp/pull/3128)
1414
* Added `dpnp.top_k` to return the `k` largest or smallest elements and their indices along an axis, with a `sorted` keyword to return them in sorted order [#3133](https://github.com/IntelPython/dpnp/pull/3133)
1515
* Added implementation of `dpnp.minmax`, computing the minimum and the maximum of an array in a single pass by a new fused reduction kernel [#3135](https://github.com/IntelPython/dpnp/pull/3135)
16+
* Added implementation of `dpnp.scipy.linalg.cho_factor` and `dpnp.scipy.linalg.cho_solve` (SciPy-compatible) [#3116](https://github.com/IntelPython/dpnp/pull/3116)
1617

1718
### Changed
1819

‎doc/reference/linalg.rst‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,6 @@ Decompositions
4343

4444
linalg.cholesky
4545
linalg.qr
46-
linalg.lu_factor
4746
linalg.svd
4847
linalg.svdvals
4948

@@ -86,7 +85,6 @@ Solving linear equations
8685
linalg.solve
8786
linalg.tensorsolve
8887
linalg.lstsq
89-
linalg.lu_solve
9088
linalg.inv
9189
linalg.pinv
9290
linalg.tensorinv

‎doc/reference/scipy_linalg.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ Decompositions
1212
:toctree: generated/
1313
:nosignatures:
1414

15+
cho_factor
16+
cho_solve
1517
lu
1618
lu_factor
1719
lu_solve

‎dpnp/libtensor/extensions/blas/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ set(_module_src
3434
${CMAKE_CURRENT_SOURCE_DIR}/gemm_batch.cpp
3535
${CMAKE_CURRENT_SOURCE_DIR}/gemv.cpp
3636
${CMAKE_CURRENT_SOURCE_DIR}/syrk.cpp
37+
${CMAKE_CURRENT_SOURCE_DIR}/trsm_batch.cpp
3738
)
3839

3940
build_dpnp_pybind11_ext(

‎dpnp/libtensor/extensions/blas/blas_py.cpp‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@
4343
#include "gemm.hpp"
4444
#include "gemv.hpp"
4545
#include "syrk.hpp"
46+
#include "trsm.hpp"
4647

4748
namespace blas_ns = dpnp::extensions::blas;
4849
namespace py = pybind11;
@@ -58,6 +59,7 @@ void init_dispatch_vectors_tables(void)
5859
blas_ns::init_gemm_dispatch_table();
5960
blas_ns::init_gemv_dispatch_vector();
6061
blas_ns::init_syrk_dispatch_vector();
62+
blas_ns::init_trsm_batch_dispatch_vector();
6163
}
6264

6365
static dot_impl_fn_ptr_t dot_dispatch_vector[dpnp_td_ns::num_types];
@@ -162,6 +164,17 @@ PYBIND11_MODULE(_blas_impl, m, py::mod_gil_not_used())
162164
py::arg("depends") = py::list());
163165
}
164166

167+
{
168+
m.def("_trsm_batch", &blas_ns::trsm_batch,
169+
"Call `trsm_batch` from oneMKL BLAS library to solve "
170+
"op(A) X = B for a batch of triangular matrices A "
171+
"(a 2-D A is shared by the batch). "
172+
"trans_op picks op(A): 0=N, 1=T, 2=C.",
173+
py::arg("sycl_queue"), py::arg("matrixA"), py::arg("matrixB"),
174+
py::arg("upper_lower"), py::arg("trans_op") = 0,
175+
py::arg("depends") = py::list());
176+
}
177+
165178
{
166179
m.def(
167180
"_using_onemath",
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
//*****************************************************************************
2+
// Copyright (c) 2026, Intel Corporation
3+
// All rights reserved.
4+
//
5+
// Redistribution and use in source and binary forms, with or without
6+
// modification, are permitted provided that the following conditions are met:
7+
// - Redistributions of source code must retain the above copyright notice,
8+
// this list of conditions and the following disclaimer.
9+
// - Redistributions in binary form must reproduce the above copyright notice,
10+
// this list of conditions and the following disclaimer in the documentation
11+
// and/or other materials provided with the distribution.
12+
// - Neither the name of the copyright holder nor the names of its contributors
13+
// may be used to endorse or promote products derived from this software
14+
// without specific prior written permission.
15+
//
16+
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
17+
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18+
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19+
// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
20+
// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
21+
// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
22+
// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
23+
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
24+
// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
25+
// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
26+
// THE POSSIBILITY OF SUCH DAMAGE.
27+
//*****************************************************************************
28+
29+
#pragma once
30+
31+
#include <cstdint>
32+
#include <utility>
33+
#include <vector>
34+
35+
#include <oneapi/mkl.hpp>
36+
#include <sycl/sycl.hpp>
37+
38+
#include "dpnp4pybind11.hpp"
39+
40+
namespace dpnp::extensions::blas
41+
{
42+
extern std::pair<sycl::event, sycl::event>
43+
trsm_batch(sycl::queue &exec_q,
44+
const dpnp::tensor::usm_ndarray &matrixA,
45+
const dpnp::tensor::usm_ndarray &matrixB,
46+
const std::int8_t upper_lower,
47+
const int trans_op,
48+
const std::vector<sycl::event> &depends);
49+
50+
extern void init_trsm_batch_dispatch_vector(void);
51+
} // namespace dpnp::extensions::blas
Lines changed: 271 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
//*****************************************************************************
2+
// Copyright (c) 2026, Intel Corporation
3+
// All rights reserved.
4+
//
5+
// Redistribution and use in source and binary forms, with or without
6+
// modification, are permitted provided that the following conditions are met:
7+
// - Redistributions of source code must retain the above copyright notice,
8+
// this list of conditions and the following disclaimer.
9+
// - Redistributions in binary form must reproduce the above copyright notice,
10+
// this list of conditions and the following disclaimer in the documentation
11+
// and/or other materials provided with the distribution.
12+
// - Neither the name of the copyright holder nor the names of its contributors
13+
// may be used to endorse or promote products derived from this software
14+
// without specific prior written permission.
15+
//
16+
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
17+
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18+
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19+
// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
20+
// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
21+
// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
22+
// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
23+
// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
24+
// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
25+
// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
26+
// THE POSSIBILITY OF SUCH DAMAGE.
27+
//*****************************************************************************
28+
29+
#include <algorithm>
30+
#include <cstdint>
31+
#include <sstream>
32+
#include <stdexcept>
33+
#include <string>
34+
#include <utility>
35+
#include <vector>
36+
37+
#include <pybind11/pybind11.h>
38+
39+
// utils extension header
40+
#include "ext/common.hpp"
41+
42+
// dpnp tensor headers
43+
#include "utils/memory_overlap.hpp"
44+
#include "utils/output_validation.hpp"
45+
#include "utils/type_utils.hpp"
46+
47+
#include "trsm.hpp"
48+
#include "types_matrix.hpp"
49+
50+
namespace dpnp::extensions::blas
51+
{
52+
namespace mkl_blas = oneapi::mkl::blas;
53+
namespace py = pybind11;
54+
namespace type_utils = dpnp::tensor::type_utils;
55+
56+
using ext::common::init_dispatch_vector;
57+
58+
typedef sycl::event (*trsm_batch_impl_fn_ptr_t)(
59+
sycl::queue &,
60+
const oneapi::mkl::uplo,
61+
const oneapi::mkl::transpose,
62+
const std::int64_t, // m
63+
const std::int64_t, // n
64+
const char *, // a
65+
const std::int64_t, // lda
66+
const std::int64_t, // stride_a
67+
char *, // b
68+
const std::int64_t, // ldb
69+
const std::int64_t, // stride_b
70+
const std::int64_t, // batch_size
71+
const std::vector<sycl::event> &);
72+
73+
static trsm_batch_impl_fn_ptr_t
74+
trsm_batch_dispatch_vector[dpnp_td_ns::num_types];
75+
76+
template <typename T>
77+
static sycl::event trsm_batch_impl(sycl::queue &exec_q,
78+
const oneapi::mkl::uplo upper_lower,
79+
const oneapi::mkl::transpose trans,
80+
const std::int64_t m,
81+
const std::int64_t n,
82+
const char *matrixA,
83+
const std::int64_t lda,
84+
const std::int64_t stride_a,
85+
char *matrixB,
86+
const std::int64_t ldb,
87+
const std::int64_t stride_b,
88+
const std::int64_t batch_size,
89+
const std::vector<sycl::event> &depends)
90+
{
91+
type_utils::validate_type_for_device<T>(exec_q);
92+
93+
const T *a = reinterpret_cast<const T *>(matrixA);
94+
T *b = reinterpret_cast<T *>(matrixB);
95+
96+
std::stringstream error_msg;
97+
bool is_exception_caught = false;
98+
99+
sycl::event trsm_batch_event;
100+
try {
101+
// Release GIL to avoid serialization of host task submissions
102+
// to the same queue in OneMKL
103+
py::gil_scoped_release lock{};
104+
105+
trsm_batch_event = mkl_blas::column_major::trsm_batch(
106+
exec_q,
107+
oneapi::mkl::side::left, // op(A) is on the left: op(A) X = B
108+
upper_lower, // Whether A is upper or lower triangular.
109+
trans, // Defines op(A): 'N' for A, 'T' for A^T, or 'C'
110+
// for A^H.
111+
oneapi::mkl::diag::nonunit, // A is not unit triangular.
112+
m, // Number of rows of B, and the order of A.
113+
n, // Number of columns of B.
114+
T(1), // Scaling factor for B.
115+
a, // Pointer to the batch of triangular matrices A.
116+
lda, // Leading dimension of each matrix A.
117+
stride_a, // Stride between consecutive A matrices.
118+
b, // Pointer to the batch of matrices B, overwritten
119+
// by the solutions X.
120+
ldb, // Leading dimension of each matrix B.
121+
stride_b, // Stride between consecutive B matrices.
122+
batch_size, // Total number of matrices in the batch.
123+
depends);
124+
} catch (oneapi::mkl::exception const &e) {
125+
error_msg << "Unexpected MKL exception caught during trsm_batch() "
126+
"call:\nreason: "
127+
<< e.what();
128+
is_exception_caught = true;
129+
} catch (sycl::exception const &e) {
130+
error_msg
131+
<< "Unexpected SYCL exception caught during trsm_batch() call:\n"
132+
<< e.what();
133+
is_exception_caught = true;
134+
}
135+
136+
if (is_exception_caught) // an unexpected error occurs
137+
{
138+
throw std::runtime_error(error_msg.str());
139+
}
140+
141+
return trsm_batch_event;
142+
}
143+
144+
std::pair<sycl::event, sycl::event>
145+
trsm_batch(sycl::queue &exec_q,
146+
const dpnp::tensor::usm_ndarray &matrixA,
147+
const dpnp::tensor::usm_ndarray &matrixB,
148+
const std::int8_t upper_lower,
149+
const int trans_op,
150+
const std::vector<sycl::event> &depends)
151+
{
152+
const int a_nd = matrixA.get_ndim();
153+
const int b_nd = matrixB.get_ndim();
154+
155+
// batch is the last dimension: (m, m, batch) and (m, n, batch),
156+
// a 2-D (m, m) matrix A is shared by all right-hand sides
157+
if ((a_nd != 2 && a_nd != 3) || b_nd != 3) {
158+
throw py::value_error(
159+
"The triangular and right-hand sides arrays must be "
160+
"2- or 3-dimensional and 3-dimensional, but got ndim=" +
161+
std::to_string(a_nd) + " and ndim=" + std::to_string(b_nd) + ".");
162+
}
163+
if (trans_op < 0 || trans_op > 2) {
164+
throw py::value_error(
165+
"trsm_batch: trans_op must be 0 (N), 1 (T), or 2 (C).");
166+
}
167+
168+
const py::ssize_t *a_shape = matrixA.get_shape_raw();
169+
const py::ssize_t *b_shape = matrixB.get_shape_raw();
170+
171+
if (a_shape[0] != a_shape[1]) {
172+
throw py::value_error("Expected batch of square matrices, but got "
173+
"matrix shape (" +
174+
std::to_string(a_shape[0]) + ", " +
175+
std::to_string(a_shape[1]) + ") in batch");
176+
}
177+
if (a_shape[0] != b_shape[0] || (a_nd == 3 && a_shape[2] != b_shape[2])) {
178+
throw py::value_error("The shapes of the triangular and right-hand "
179+
"sides arrays are incompatible.");
180+
}
181+
182+
// check compatibility of execution queue and allocation queue
183+
if (!dpnp::utils::queues_are_compatible(exec_q, {matrixA, matrixB})) {
184+
throw py::value_error(
185+
"Execution queue is not compatible with allocation queues");
186+
}
187+
188+
auto const &overlap = dpnp::tensor::overlap::MemoryOverlap();
189+
if (overlap(matrixA, matrixB)) {
190+
throw py::value_error("The triangular and right-hand sides arrays "
191+
"are overlapping segments of memory");
192+
}
193+
194+
dpnp::tensor::validation::CheckWritable::throw_if_not_writable(matrixB);
195+
196+
if (!matrixA.is_f_contiguous() || !matrixB.is_f_contiguous()) {
197+
throw py::value_error("The triangular and right-hand sides arrays "
198+
"must be F-contiguous");
199+
}
200+
201+
auto array_types = dpnp_td_ns::usm_ndarray_types();
202+
const int a_type_id =
203+
array_types.typenum_to_lookup_id(matrixA.get_typenum());
204+
const int b_type_id =
205+
array_types.typenum_to_lookup_id(matrixB.get_typenum());
206+
207+
if (a_type_id != b_type_id) {
208+
throw py::value_error("The types of the triangular and right-hand "
209+
"sides arrays are mismatched");
210+
}
211+
212+
trsm_batch_impl_fn_ptr_t trsm_batch_fn =
213+
trsm_batch_dispatch_vector[a_type_id];
214+
if (trsm_batch_fn == nullptr) {
215+
throw py::value_error(
216+
"No trsm_batch implementation defined for the provided type "
217+
"of the input matrix");
218+
}
219+
220+
const std::int64_t m = a_shape[0];
221+
const std::int64_t n = b_shape[1];
222+
const std::int64_t batch_size = b_shape[2];
223+
224+
// nothing to solve, oneMKL does not need to be called
225+
if (m == 0 || n == 0 || batch_size == 0) {
226+
return std::make_pair(sycl::event(), sycl::event());
227+
}
228+
229+
const std::int64_t lda = std::max<std::int64_t>(1, m);
230+
const std::int64_t ldb = std::max<std::int64_t>(1, m);
231+
// stride 0 reuses the shared matrix A for each batch element
232+
const std::int64_t stride_a = (a_nd == 2) ? 0 : m * m;
233+
const std::int64_t stride_b = m * n;
234+
235+
const oneapi::mkl::uplo uplo_val =
236+
static_cast<oneapi::mkl::uplo>(upper_lower);
237+
const oneapi::mkl::transpose trans =
238+
(trans_op == 0) ? oneapi::mkl::transpose::N
239+
: (trans_op == 1) ? oneapi::mkl::transpose::T
240+
: oneapi::mkl::transpose::C;
241+
242+
sycl::event trsm_batch_ev = trsm_batch_fn(
243+
exec_q, uplo_val, trans, m, n, matrixA.get_data(), lda, stride_a,
244+
matrixB.get_data(), ldb, stride_b, batch_size, depends);
245+
246+
sycl::event args_ev = dpnp::utils::keep_args_alive(
247+
exec_q, {matrixA, matrixB}, {trsm_batch_ev});
248+
249+
return std::make_pair(args_ev, trsm_batch_ev);
250+
}
251+
252+
template <typename fnT, typename T>
253+
struct TrsmBatchContigFactory
254+
{
255+
fnT get()
256+
{
257+
if constexpr (types::TrsmBatchTypePairSupportFactory<T>::is_defined) {
258+
return trsm_batch_impl<T>;
259+
}
260+
else {
261+
return nullptr;
262+
}
263+
}
264+
};
265+
266+
void init_trsm_batch_dispatch_vector(void)
267+
{
268+
init_dispatch_vector<trsm_batch_impl_fn_ptr_t, TrsmBatchContigFactory>(
269+
trsm_batch_dispatch_vector);
270+
}
271+
} // namespace dpnp::extensions::blas

0 commit comments

Comments
 (0)