text
🚨 New Release of 🤗PEFT!
🚨 New Release of 🤗PEFT!
new_line
new_line
text
1. New methods for merging LoRA weights. Refer this HF Post for more details: 
1. New methods for merging LoRA weights. Refer this HF Post for more details: 
link
https://huggingface.co/posts/smangrul/850816632583824
https://huggingface.co/posts/smangrul/850816632583824
new_line
new_line
text
2. AWQ and AQLM support for LoRA. You can now:
2. AWQ and AQLM support for LoRA. You can now:
new_line
text
- Train adapters on top of 2-bit quantized models with AQLM
- Train adapters on top of 2-bit quantized models with AQLM
new_line
text
- Train adapters on top of powerful AWQ quantized models
- Train adapters on top of powerful AWQ quantized models
new_line
text
Note for inference you can't merge the LoRA weights into the base model!
Note for inference you can't merge the LoRA weights into the base model!
new_line
new_line
text
3. DoRA support: Enabling DoRA is as easy as adding 
3. DoRA support: Enabling DoRA is as easy as adding 
inline_code
`use_dora=True`
use_dora=True
text
 to your 
 to your 
inline_code
`LoraConfig`
LoraConfig
text
. Find out more about this method here: 
. Find out more about this method here: 
link
https://arxiv.org/abs/2402.09353
https://arxiv.org/abs/2402.09353
new_line
new_line
text
4. Improved documentation, particularly docs regarding PEFT LoRA+DeepSpeed and PEFT LoRA+FSDP! 📄 Check out the docs at 
4. Improved documentation, particularly docs regarding PEFT LoRA+DeepSpeed and PEFT LoRA+FSDP! 📄 Check out the docs at 
link
https://huggingface.co/docs/peft/index
https://huggingface.co/docs/peft/index
text
.
.
new_line
new_line
text
5. Full Release Notes: 
5. Full Release Notes: 
link
https://github.com/huggingface/peft/releases/tag/v0.9.0
https://github.com/huggingface/peft/releases/tag/v0.9.0
new_line
🚨 New Release of 🤗PEFT!

1. New methods for merging LoRA weights. Refer this HF Post for more details: https://huggingface.co/posts/smangrul/850816632583824

2. AWQ and AQLM support for LoRA. You can now:
- Train adapters on top of 2-bit quantized models with AQLM
- Train adapters on top of powerful AWQ quantized models
Note for inference you can't merge the LoRA weights into the base model!

3. DoRA support: Enabling DoRA is as easy as adding `use_dora=True` to your `LoraConfig`. Find out more about this method here: https://arxiv.org/abs/2402.09353

4. Improved documentation, particularly docs regarding PEFT LoRA+DeepSpeed and PEFT LoRA+FSDP! 📄 Check out the docs at https://huggingface.co/docs/peft/index.

5. Full Release Notes: https://github.com/huggingface/peft/releases/tag/v0.9.0

❤️
osanseviero Kukedlc clem Ji-Ha samusenps alielfilali01 mudogruer ybelkada ryokeken Cstark dahwinsingularity
👍
NickyNicky Jenish-23 ixaxaar ababio rizwan-ai ybelkada Cstark ZennyKenny catastropiyush
/avatars/c02f9f8d27cb83be65dba0c7b945daa4.svg
Jenish-23
Jenish-23
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1638132956881-5fca176d1d7a08cb34d79d5d.jpeg
Sourab Mangrulkar
smangrul
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1648631057413-noauth.png
Younes Belkada
ybelkada
user
https://cdn-avatars.huggingface.co/v1/production/uploads/626237d9bbcbd1c34f1bb231/EJrOjvAL-68qMCYdnvOrq.png
Ali El Filali
alielfilali01
user