

Methods and tools for efficient training on a single GPU: start here to learn common approaches that can help optimize GPU memory utilization, speed up the training, or both. 
Multi-GPU training section: explore this section to learn about further optimization methods that apply to a multi-GPU settings, such as data, tensor, and pipeline parallelism.
CPU training section: learn about mixed precision training on CPU.
Efficient Training on Multiple CPUs: learn about distributed CPU training.
Training on TPU with TensorFlow: if you are new to TPUs, refer to this section for an opinionated introduction to training on TPUs and using XLA. 
Custom hardware for training: find tips and tricks when building your own deep learning rig.
Hyperparameter Search using Trainer API