text
QuIP# ecosystem is growing :) 
QuIP# ecosystem is growing :) 
new_line
new_line
text
I've seen a quip# 2 bit Qwen-72b-Chat model today on the hub that shows there is support for vLLM inference. 
I've seen a quip# 2 bit Qwen-72b-Chat model today on the hub that shows there is support for vLLM inference. 
new_line
text
This will speed up inference and make high performing 2 bit models more practical. I'm considering quipping MoMo now, as I can only use brief context window of Qwen-72b on my system otherwise, even with bnb double quantization. 
This will speed up inference and make high performing 2 bit models more practical. I'm considering quipping MoMo now, as I can only use brief context window of Qwen-72b on my system otherwise, even with bnb double quantization. 
new_line
new_line
resource
https://huggingface.co/keyfan/Qwen-72B-Chat-2bit
model
keyfan/Qwen-72B-Chat-2bit
https://huggingface.co/keyfan/Qwen-72B-Chat-2bit
new_line
new_line
text
Also notice the easier to use Quip# for all library :) 
Also notice the easier to use Quip# for all library :) 
new_line
new_line
link
https://github.com/chu-tianxiang/QuIP-for-all
https://github.com/chu-tianxiang/QuIP-for-all
QuIP# ecosystem is growing :) 

I've seen a quip# 2 bit Qwen-72b-Chat model today on the hub that shows there is support for vLLM inference. 
This will speed up inference and make high performing 2 bit models more practical. I'm considering quipping MoMo now, as I can only use brief context window of Qwen-72b on my system otherwise, even with bnb double quantization. 

https://huggingface.co/keyfan/Qwen-72B-Chat-2bit

Also notice the easier to use Quip# for all library :) 

https://github.com/chu-tianxiang/QuIP-for-all
🤗
Felladrin osanseviero victor ybelkada reach-vb merve samusenps mexicanamerican
👍
osanseviero ybelkada reach-vb Tonic kramp merve abidlabs
❤️
osanseviero ybelkada reach-vb merve samusenps sbrandeis
🤯
reach-vb merve sbrandeis
https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg
Julien Chaumond
julien-c
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1608146735109-5fcfb7c407408029ba3577e2.png
Simon Brandeis
sbrandeis
user