How to install flash attention on HF gradio space
I tried to putflash-attnin the requirements.txt file to install flash-attention on my space, but it gives error that torch is not installed.I also tried to puttorchaboveflash-attnbut still couldn’t, probably torch is not installed yet.Please help!
2024-01-30T18:25:21Z
hi@nxphi47,One option is to use a custom Dockerfile and install as a build stephuggingface.coDockerfile · ylacombe/children-story at mainWe’re on a journey to advance and democratize artificial intelligence through open source and open science.Another option if you’re using Gradio/Streamlit SDK is to install at runtimeimport subprocess
subprocess.run('pip install flash-attn --no-build-isolation', env={'FLASH_ATTENTION_SKIP_CUDA_BUILD': "TRUE"}, shell=True)
2024-06-16T15:51:43Z
put this prebuilt whl inside your requirements.txthttps://github.com/Dao-AILab/flash-attention/releases/download/v2.5.9.post1/flash_attn-2.5.9.post1+cu118torch1.12cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
2024-06-17T19:22:39Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.