Quants to fit into 48GB with 16k context
#2
by
Abalagaev
- opened
Will there be small-weight quants of this model? 8b model is fantastic, but requires additional LLM for reasoning on complex issues the model has figured out. Qwen2.5-72B in Q3_K_S reasoning is quite good.