bartowski

/

Meta-Llama-3.1-8B-Instruct-GGUF

like 162

hi bartowski 👋, i wasn't sure where to ask, but given that it's a based on LLaMA-3.1-8B I think this might be the best place.

i'm really blown back by this model: HuatuoGPT-o1-8B.

i think it might be the first implementation of an 8b reasoning model that actually works.

i've noticed that the other small reasoning models right now seem to kind of follow the reasoning template in a forced way, but this one actually comes to valid conclusions, if that makes sense?

its a model for medical questions/reasoning.

and so i wanted to kindly ask you if its possible to make gguf quants for that model? thank you a lot regardless for all your work, really appreciate it.