Llama2 70b - Cuda out of memory exceptions
Hello,I have 2 GPU of 24 GB RTX 4090 GPU.I want to fine-tune the 70b model but it throws acuda out of memory exceptionseven though I have used Lora and BitsAndBytesConfig.Let me know if I’m overlooking this or please give me suggestions.Thanks.