⚡️ Inference

A blog post on How 🤗 Accelerate runs very large models thanks to PyTorch with OPT.

Combining OPT and Flash Attention 2
First, make sure to install the latest version of Flash Attention 2 to include the sliding window attention feature.