accelerate launch --num_processes 2 trainer-program.py 

Use --num_gpus to select how many GPUs to use.

deepspeed --num_gpus 2 trainer-program.py 

Order of GPUs
Now, to select which GPUs to use and their order, you'll use the CUDA_VISIBLE_DEVICES environment variable. It is easiest to set the environment variable in a ~/bashrc or another startup config file. CUDA_VISIBLE_DEVICES is used to map which GPUs are used. For example, if you have 4 GPUs (0, 1, 2, 3) and you only want to run GPUs 0 and 2: