How was this quantized?
#3
by
imq
- opened
When I load the safetensor original of deepseek R1 Distill LLama 8B and use 8 bit in text-generation-webui with the transformers loader the quality of the output is a lot better than with this 8 bit gguf. When I ask how many "r" characters are in strawberry it spells it like this stravberry and miscounts the number from 2, 4 or 5. With the original but loaded in 8 bit it's correctly spelling and counting the "r" characters.