Skip to content

Reduce testsuite runtime by merging similar reduction testsets#728

Open
christiangnrd wants to merge 2 commits into
mainfrom
mergetests
Open

Reduce testsuite runtime by merging similar reduction testsets#728
christiangnrd wants to merge 2 commits into
mainfrom
mergetests

Conversation

@christiangnrd
Copy link
Copy Markdown
Member

@christiangnrd christiangnrd commented Jun 3, 2026

Merge mapreducedim! and reducedim! testsets.
Also merge mapreduce and reduce testsets while factoring out the &, |, and xor tests to keep Alloc values reasonable.

The result is that 4 testsets that used to take well over a minute each are now 2 testsets that take a few seconds longer than the longest of the 2 and a shorter ~30s testset.

Probably not a huge difference tests are mostly run in parallel, but this feels like a huge amount of time that used to be wasted compiling kernels instead of actually testing.

Timing comparison

Command: using Pkg; Pkg.test("CUDA|Metal"; test_args=["--jobs=1", "--verbose", "gpuarrays/reductions/mapred", "gpuarrays/reductions/and", "gpuarrays/reductions/red"])
*I also forced a new worker for every test in runtest.jl's test_worker to ensure worst-case scenario

# metal before
                                               │   Test   │   Init   │ Compile │ ──────────────── CPU ──────────────── │
Test                                  (Worker) │ time (s) │ time (s) │   (%)   │ GC (s) │ GC % │ Alloc (MB) │ RSS (MB) │
gpuarrays/reductions/mapreduce             (1) │        started at 2026-06-03T16:58:12.697
gpuarrays/reductions/mapreduce             (1) │    80.19 │     7.50 │   76.16 │   2.82 │  3.5 │   22161.98 │  1305.05 │
gpuarrays/reductions/mapreducedim!_large   (2) │        started at 2026-06-03T16:59:40.564
gpuarrays/reductions/mapreducedim!_large   (2) │    28.38 │     7.26 │   73.63 │   1.17 │  4.1 │    7607.99 │  1047.58 │
gpuarrays/reductions/mapreducedim!         (3) │        started at 2026-06-03T17:00:16.234
gpuarrays/reductions/mapreducedim!         (3) │    77.90 │     7.10 │   76.32 │   2.90 │  3.7 │   22157.35 │  1300.23 │
gpuarrays/reductions/reducedim!            (4) │        started at 2026-06-03T17:01:41.759
gpuarrays/reductions/reducedim!            (4) │    56.79 │     7.30 │   79.78 │   2.06 │  3.6 │   15733.63 │  1159.98 │
gpuarrays/reductions/reduce                (5) │        started at 2026-06-03T17:02:46.027
gpuarrays/reductions/reduce                (5) │    78.27 │     7.28 │   76.85 │   2.98 │  3.8 │   22238.45 │  1332.27 │

Test Summary:                                | Pass  Total     Time
  Overall                                    | 1349   1349  6m00.1s

#metal now
                                               │   Test   │   Init   │ Compile │ ──────────────── CPU ──────────────── │
Test                                  (Worker) │ time (s) │ time (s) │   (%)   │ GC (s) │ GC % │ Alloc (MB) │ RSS (MB) │
gpuarrays/reductions/mapreduce             (1) │        started at 2026-06-03T16:53:11.516
gpuarrays/reductions/mapreduce             (1) │    85.43 │     7.34 │   75.85 │   3.18 │  3.7 │   22905.00 │  1405.03 │
gpuarrays/reductions/mapreducedim!         (2) │        started at 2026-06-03T16:54:45.118
gpuarrays/reductions/mapreducedim!         (2) │    87.28 │     7.71 │   76.35 │   3.14 │  3.6 │   22182.17 │  1312.38 │
gpuarrays/reductions/mapreducedim!_large   (3) │        started at 2026-06-03T16:56:20.193
gpuarrays/reductions/mapreducedim!_large   (3) │    30.06 │     7.47 │   74.45 │   1.38 │  4.6 │    7608.21 │  1032.05 │
gpuarrays/reductions/and or xor            (4) │        started at 2026-06-03T16:56:57.955
gpuarrays/reductions/and or xor            (4) │    27.55 │     7.43 │   80.29 │   1.09 │  4.0 │    7121.14 │   918.94 │

Test Summary:                                | Pass  Total     Time
  Overall                                    | 1349   1349  4m22.2s

# CUDA before
                                               │   Test   │   Init   │ Compile │ ──────────── GPU ───────────── │ ──────────────── CPU ──────────────── │
Test                                  (Worker) │ time (s) │ time (s) │   (%)   │ GC (s) │ Alloc (MB) │ RSS (MB) │ GC (s) │ GC % │ Alloc (MB) │ RSS (MB) │
gpuarrays/reductions/reducedim!            (1) │        started at 2026-06-03T17:00:18.658
gpuarrays/reductions/reducedim!            (1) │    84.29 │    13.23 │    0.00 │   0.01 │       1.03 │   150.00 │   2.11 │  2.5 │    8011.49 │  2269.91 │
gpuarrays/reductions/reduce                (2) │        started at 2026-06-03T17:01:57.853
gpuarrays/reductions/reduce                (2) │   102.86 │    12.91 │    0.00 │   0.02 │       1.51 │   150.00 │   2.43 │  2.4 │   10534.54 │  2295.56 │
gpuarrays/reductions/mapreducedim!_large   (3) │        started at 2026-06-03T17:03:55.405
gpuarrays/reductions/mapreducedim!_large   (3) │    39.66 │    12.89 │    0.00 │   0.01 │     186.05 │   182.00 │   1.46 │  3.7 │    4230.51 │  2039.38 │
gpuarrays/reductions/mapreduce             (4) │        started at 2026-06-03T17:04:49.450
gpuarrays/reductions/mapreduce             (4) │   102.99 │    12.76 │    0.00 │   0.02 │       1.81 │   152.00 │   2.44 │  2.4 │   10307.30 │  2305.18 │
gpuarrays/reductions/mapreducedim!         (5) │        started at 2026-06-03T17:06:47.203
gpuarrays/reductions/mapreducedim!         (5) │   104.98 │    12.90 │    0.00 │   0.01 │       1.54 │   152.00 │   2.38 │  2.3 │    9639.75 │  2374.53 │

Test Summary:                                | Pass  Total     Time
  Overall                                    | 1553   1553  8m29.6s

# CUDA now
                                               │   Test   │   Init   │ Compile │ ──────────── GPU ───────────── │ ──────────────── CPU ──────────────── │
Test                                  (Worker) │ time (s) │ time (s) │   (%)   │ GC (s) │ Alloc (MB) │ RSS (MB) │ GC (s) │ GC % │ Alloc (MB) │ RSS (MB) │
gpuarrays/reductions/mapreducedim!         (1) │        started at 2026-06-03T17:15:42.413
gpuarrays/reductions/mapreducedim!         (1) │   105.67 │    13.29 │    0.00 │   0.01 │       2.57 │   152.00 │   2.36 │  2.2 │    9657.06 │  2390.07 │
gpuarrays/reductions/mapreducedim!_large   (2) │        started at 2026-06-03T17:17:42.935
gpuarrays/reductions/mapreducedim!_large   (2) │    39.60 │    12.79 │    0.00 │   0.01 │     186.05 │   182.00 │   1.45 │  3.7 │    4230.55 │  2039.03 │
gpuarrays/reductions/and or xor            (3) │        started at 2026-06-03T17:18:36.990
gpuarrays/reductions/and or xor            (3) │    30.32 │    12.85 │    0.00 │   0.01 │       0.30 │   148.00 │   0.71 │  2.3 │    3126.03 │  1939.61 │
gpuarrays/reductions/mapreduce             (4) │        started at 2026-06-03T17:19:21.736
gpuarrays/reductions/mapreduce             (4) │   105.53 │    12.84 │    0.00 │   0.02 │       3.02 │   152.00 │   2.48 │  2.4 │   10667.96 │  2310.68 │

Test Summary:                                | Pass  Total     Time
  Overall                                    | 1553   1553  5m41.0s

@christiangnrd
Copy link
Copy Markdown
Member Author

Test failures unrelated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant