Use the --nproc_per_node to select how many GPUs to use.

torchrun --nproc_per_node=2  trainer-program.py 

Use --num_processes to select how many GPUs to use.

accelerate launch --num_processes 2 trainer-program.py 

Use --num_gpus to select how many GPUs to use.

deepspeed --num_gpus 2 trainer-program.py