text
Try out Mixtral 2-bit on a free-tier Google Colab notebook right now! 
Try out Mixtral 2-bit on a free-tier Google Colab notebook right now! 
new_line
new_line
link
https://colab.research.google.com/drive/1-xZmBRXT5Fm3Ghn4Mwa2KRypORXb855X?usp=sharing
https://colab.research.google.com/drive/1-xZmBRXT5Fm3Ghn4Mwa2KRypORXb855X?usp=sharing
text
new_line
new_line
text
AQLM method has been recently introduced on transformers main branch 
AQLM method has been recently introduced on transformers main branch 
new_line
new_line
text
The 2bit model can be found here: 
The 2bit model can be found here: 
resource
https://huggingface.co/BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch
model
BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch
https://huggingface.co/BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch
text
new_line
new_line
text
And you can read more about the method here: 
And you can read more about the method here: 
link
https://huggingface.co/docs/transformers/main/en/quantization#aqlm
https://huggingface.co/docs/transformers/main/en/quantization#aqlm
new_line
new_line
text
Great work 
Great work 
mention
@BlackSamorez
BlackSamorez
text
 and team!
 and team!
Try out Mixtral 2-bit on a free-tier Google Colab notebook right now! 

https://colab.research.google.com/drive/1-xZmBRXT5Fm3Ghn4Mwa2KRypORXb855X?usp=sharing 

AQLM method has been recently introduced on transformers main branch 

The 2bit model can be found here: https://huggingface.co/BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch 

And you can read more about the method here: https://huggingface.co/docs/transformers/main/en/quantization#aqlm

Great work @BlackSamorez and team!
❤️
osanseviero alielfilali01 sayhan Chunte victor samusenps macadeliccc clem Dlbk andysalerno jeffboudier loubnabnl linoyts medmac01 mathiasn1 Ji-Ha kramp bombassaro
/avatars/3fb9c2725e25fc91b6f72a41f632c8d0.svg
Vishvesh Mangal
Kingvishy
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1648631057413-noauth.png
Younes Belkada
ybelkada
user
/avatars/a5d3be1c3e9df8a367922d7aa3214cca.svg
Kevin Amiri
kevin009
user
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/EExvesA97-cULo70CK_gj.jpeg
koo
K00B404
user