FlashAttention-2 is currently supported for the following architectures:
* Bark
* Bart
* DistilBert
* Gemma
* GPTBigCode
* GPTNeo
* GPTNeoX
* Falcon
* Llama
* Llava
* VipLlava
* MBart
* Mistral
* Mixtral
* OPT
* Phi
* StableLm
* Starcoder2
* Qwen2
* Whisper
You can request to add FlashAttention-2 support for another model by opening a GitHub Issue or Pull Request.
Before you begin, make sure you have FlashAttention-2 installed.