Follow along with the more in-depth Accelerate guide for FSDP.
Read the Introducing PyTorch Fully Sharded Data Parallel (FSDP) API blog post.
Read the Scaling PyTorch models on Cloud TPUs with FSDP blog post.