text
Check out quantized weights from ISTA-DAS Lab directly in their organisation page: 
Check out quantized weights from ISTA-DAS Lab directly in their organisation page: 
link
https://huggingface.co/ISTA-DASLab
https://huggingface.co/ISTA-DASLab
text
 ! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)
 ! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)
new_line
new_line
text
Read more about these techniques below:
Read more about these techniques below:
new_line
new_line
text
AQLM paper: 
AQLM paper: 
resource
https://hf.co/papers/2401.06118
paper
2401.06118
https://hf.co/papers/2401.06118
Extreme Compression of Large Language Models via Additive Quantization (2401.06118)
new_line
text
QMoE: 
QMoE: 
resource
https://huggingface.co/papers/2310.16795
paper
2310.16795
https://huggingface.co/papers/2310.16795
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models (2310.16795)
new_line
new_line
text
Some useful links below:
Some useful links below:
new_line
new_line
text
AQLM repo: 
AQLM repo: 
link
https://github.com/Vahe1994/AQLM
https://github.com/Vahe1994/AQLM
new_line
text
How to use AQLM & transformers: 
How to use AQLM & transformers: 
link
https://huggingface.co/docs/transformers/quantization#aqlm
https://huggingface.co/docs/transformers/quantization#aqlm
new_line
text
How to use AQLM & PEFT: 
How to use AQLM & PEFT: 
link
https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion
https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion
new_line
new_line
text
Great work from 
Great work from 
mention
@BlackSamorez
BlackSamorez
text
 and team !
 and team !
Check out quantized weights from ISTA-DAS Lab directly in their organisation page: https://huggingface.co/ISTA-DASLab ! With official weights of AQLM (for 2bit quantization) & QMoE (1-bit MoE quantization)

Read more about these techniques below:

AQLM paper: https://hf.co/papers/2401.06118
QMoE: https://huggingface.co/papers/2310.16795

Some useful links below:

AQLM repo: https://github.com/Vahe1994/AQLM
How to use AQLM & transformers: https://huggingface.co/docs/transformers/quantization#aqlm
How to use AQLM & PEFT: https://huggingface.co/docs/peft/developer_guides/quantization#aqlm-quantizaion

Great work from @BlackSamorez and team !
❤️
alielfilali01 marcsun13 osanseviero samusenps FM-1976 mayank-mishra clem goodgame dillfrescott
👍
goodgame dillfrescott