Shouldn't `_flash_attn_2_enabled` be documented?
While reading the Llama code, I found out that we can use flash attention via optionflash_attn_2_enabledatthese lines. However, this can not be seen inLlamaConfig.After bit googling, I think to use flash attention we needDao-AILab/flash-attentionright?
2023-11-30T17:24:24Z
Hi@dinhanhx, we integrated flashattention2 recently. You can learn more about the integrationhere.Notethat not every model supports flashattention2.