Computing with GPUs

Requesting GPUs

GPUs are requested via the GRES option:

sbatch --gres=gpu ...

Request multiple GPUs with:

sbatch --gres=gpu:2

At this time we have one GPU per node.

Nodes with GPUs

Location	Partition	Node Name	GPU
FHCRC	largenode	j	NVIDIA
GCP	gpu	h	1 X Nvidia V100

Requesting GPUs on Gizmo

The GPUs are currently installed only on nodes in the largenode partition, thus it is necessary to request the minimums required for largenode jobs:

sbatch -p largenode --mem=21500 -c 6 --gres=gpu ...

Requesting GPUs on Koshu

The GPU configured nodes on koshu are in the partition “gpu”. Jobs requiring GPUs need to request this partition and the GPUs:

sbatch -p gpu --gres=gpu ...

Using GPUs

When your job is assigned a GPU, Slurm sets the environment variable CUDA_VISIBLE_DEVICES. This environment variable indicates the assigned GPU- most CUDA tools (e.g. tensorflow) use this to restrict execution to that device.