CUDA_VISIBLE_DEVICES=0,2 torchrun trainer-program.py 
Only the 2 physical GPUs (0 and 2) are "visible" to PyTorch and these are mapped to cuda:0 and cuda:1 respectively. You can also reverse the order of the GPUs to use 2 first. Now, the mapping is cuda:1 for GPU 0 and cuda:0 for GPU 2.

CUDA_VISIBLE_DEVICES=2,0 torchrun trainer-program.py 
You can also set the CUDA_VISIBLE_DEVICES environment variable to an empty value to create an environment without GPUs.