AWQ quantized version of Llama 3 8B ChatQA
Hey guys, I just quantized Llama 3 ChatQA finetuned model to 4 bit AWQ.I thought it would be useful for people as there isn’t a good chat fine-tuned version of Llama 3. Do check it out.Sreenington/Llama-3-8B-ChatQA-AWQ