Mistral from Huggingface is slow
I was trying to run a distillation on the Mistral model. I am using the Huggingface version with flash attention. The model is running exceptionally slow. The GPUs are stuck at 0% utilization. Is it normal?Screen Shot 2023-11-20 at 5.05.53 AM732×268 12.9 KB