[ZacKienzle2]

Proposed Changes

The SLURM branch of cluster_set_resource_string in batch_job/src/batch_queue_cluster.c passes the memory, wall time and cores of a task to sbatch, but not its GPUs. A Makeflow rule or a vine_factory --gpus worker that declares GPUs is therefore submitted without a GPU request, and on a cluster that confines devices with cgroups the job runs on a node where it sees no GPU.

This adds --gpus=N when the task declares GPUs, in the same way as the other resources. --gpus is the total number of GPUs that sbatch(1) documents for a job. The HTCondor module already requests GPUs the same way (request_gpus in batch_queue_condor.c).

Checked on a Slurm cluster with a one-rule JX workflow declaring "gpus": 1, run with makeflow --jx -T slurm, with no GPU option in -B:

• sacct shows ReqTRES and AllocTRES with gres/gpu=1.
• nvidia-smi -L inside the job lists the allocated GPU.

Merge Checklist

• make test Run local tests prior to pushing. (0 of 128 tests failed, 9 skipped, built with packaging/build-conda.)
• make format Format source code to comply with lint policies.
• make lint Run lint on source code prior to pushing.
• Manual Update: The SLURM section of the Makeflow manual lists the sbatch options as -N 1 -n 1 -c ${CORES} --mem=${MEMORY}M --time=${WALL_TIME_in_minutes}; it could add --gpus=${GPUS}, which this change passes when a rule requests GPUs. Happy to add it if you would like it in this PR.
• Type Labels: bugfix
• Product Labels: Makeflow, TaskVine
• PR RTM: Mark your PR as ready to merge.

[btovar]

Thanks @ZacKienzle2!