Downloads Model Every Time
The model is “meta-llama/Llama-2-7b-chat-hf”Im using the sample code, that queries the model with “I liked “Breaking Bad” and “Band of Brothers”…”When the space runs it downloads the 9GB+ model, taking a while and then outputs a response.But if I refresh the page or use the embed code the whole process of downloading the model starts again.I am at a loss to understand why this happens, I have even upgraded to persistent storage.
2023-08-24T15:48:07Z
What space sdk are you using?
2023-08-24T15:59:58Z
Streamlit.I just read the section about setting the HF_HOME variable so restarting and trying that.
2023-08-25T08:27:04Z
OK I now have the model persist on disk stopping it from having to be downloaded everytime.Im running on an A10G and it seems that every time I refresh the page the model needs to be loaded into the GPU memory, which takes a few minutes.Is there a way to persist the model in memory, I have set the gpu to sleep after an hour.
2023-11-02T22:52:44Z
mattsputnikdigital:HF_HOMEWhat was the specific setting you made? I’m having the same issue with the model downloading every time I restart.