Run a script with 🤗 Accelerate
🤗 Accelerate is a PyTorch-only library that offers a unified method for training a model on several types of setups (CPU-only, multiple GPUs, TPUs) while maintaining complete visibility into the PyTorch training loop. Make sure you have 🤗 Accelerate installed if you don't already have it:

Note: As Accelerate is rapidly developing, the git version of accelerate must be installed to run the scripts

pip install git+https://github.com/huggingface/accelerate