text
🤗 PEFT v0.10.0 release! 🔥🚀✨
🤗 PEFT v0.10.0 release! 🔥🚀✨
new_line
new_line
text
Some highli📝ghts:
Some highli📝ghts:
new_line
text
1. FSDP+QLoRA and DeepSpeed Stage-3+QLoRA
1. FSDP+QLoRA and DeepSpeed Stage-3+QLoRA
new_line
text
2. Layer expansion + LoRA
2. Layer expansion + LoRA
new_line
text
3. DoRA support for Conv2D layers and quantized bitsandbytes layers
3. DoRA support for Conv2D layers and quantized bitsandbytes layers
new_line
text
4. New LoftQ utility
4. New LoftQ utility
new_line
text
5. Batched inference for mixed LoRA adapters.
5. Batched inference for mixed LoRA adapters.
new_line
new_line
link
http://Answer.AI
http://Answer.AI
text
 team in collaboration with bitsandbytes and Hugging Face 🤗 open sourced code enabling the usage of FSDP+QLoRA and explained the whole process in their insightful blogpost 
 team in collaboration with bitsandbytes and Hugging Face 🤗 open sourced code enabling the usage of FSDP+QLoRA and explained the whole process in their insightful blogpost 
link
https://lnkd.in/g6jgfXyv
https://lnkd.in/g6jgfXyv
text
. This is now integrated into Hugging Face ecosystem.
. This is now integrated into Hugging Face ecosystem.
new_line
new_line
text
For an end-to-end example on FSDP+QLoRA, please refer 
For an end-to-end example on FSDP+QLoRA, please refer 
link
https://lnkd.in/gT3yY-Rx
https://lnkd.in/gT3yY-Rx
text
.
.
new_line
new_line
text
For an end-to-end example on DeepSpeed Stage-3+QLoRA, please refer 
For an end-to-end example on DeepSpeed Stage-3+QLoRA, please refer 
link
https://lnkd.in/gkt-xZRE
https://lnkd.in/gkt-xZRE
text
.
.
new_line
new_line
text
With the PR 
With the PR 
link
https://lnkd.in/g5F348MN
https://lnkd.in/g5F348MN
text
 these changes are now upstreamed in 
 these changes are now upstreamed in 
link
https://lnkd.in/g5_MxYtY
https://lnkd.in/g5_MxYtY
text
 thanks to Wing Lian ! 🚀
 thanks to Wing Lian ! 🚀
new_line
new_line
text
Kudos to 
Kudos to 
link
http://Answer.AI
http://Answer.AI
text
 team,  Titus von Köller , Younes Belkada, Benjamin Bossan and Zachary Mueller for all the help without which this couldn't have been possible. 🤗
 team,  Titus von Köller , Younes Belkada, Benjamin Bossan and Zachary Mueller for all the help without which this couldn't have been possible. 🤗
new_line
new_line
text
For efficient depthwise layer expansion akin to 
For efficient depthwise layer expansion akin to 
inline_code
`passthrough`
passthrough
text
 method of 
 method of 
inline_code
`mergekit`
mergekit
text
 but without using additional memory and attaching LoRAs to it, refer to the details below! 🔥https://lnkd.in/ge95ztjA
 but without using additional memory and attaching LoRAs to it, refer to the details below! 🔥https://lnkd.in/ge95ztjA
new_line
new_line
text
Now DoRA is supported for Conv2D layers as well as bitsandbytes quantized layers ✨. For more details, please refer the below thread.
Now DoRA is supported for Conv2D layers as well as bitsandbytes quantized layers ✨. For more details, please refer the below thread.
new_line
link
https://lnkd.in/gsJbuWPD
https://lnkd.in/gsJbuWPD
new_line
new_line
text
Now you can mix different LoRA adapters in a batch during inference which speeds-up the inference by avoiding computation of base model multiple times which would be the case for adaptive inference with batch_size=1! ⚡️.
Now you can mix different LoRA adapters in a batch during inference which speeds-up the inference by avoiding computation of base model multiple times which would be the case for adaptive inference with batch_size=1! ⚡️.
new_line
text
Details below. 
Details below. 
link
https://lnkd.in/gD-pcX_B
https://lnkd.in/gD-pcX_B
new_line
new_line
text
LoftQ reduces quantization error by appropriately initializing the LoRA adapter weights. Normally, this is a two-step process. Benjamin Bossan
LoftQ reduces quantization error by appropriately initializing the LoRA adapter weights. Normally, this is a two-step process. Benjamin Bossan
new_line
text
added new util 
added new util 
inline_code
`replace_lora_weights_loftq`
replace_lora_weights_loftq
text
 for LoftQ to use it on the fly with bnb.
 for LoftQ to use it on the fly with bnb.
new_line
new_line
text
For more details, refer to the release notes. 📝
For more details, refer to the release notes. 📝
new_line
link
https://lnkd.in/gg7-AmHA
https://lnkd.in/gg7-AmHA
text
. As always, make sure losses go down and be happy to watch your model train!
. As always, make sure losses go down and be happy to watch your model train!
🤗 PEFT v0.10.0 release! 🔥🚀✨

Some highli📝ghts:
1. FSDP+QLoRA and DeepSpeed Stage-3+QLoRA
2. Layer expansion + LoRA
3. DoRA support for Conv2D layers and quantized bitsandbytes layers
4. New LoftQ utility
5. Batched inference for mixed LoRA adapters.

http://Answer.AI team in collaboration with bitsandbytes and Hugging Face 🤗 open sourced code enabling the usage of FSDP+QLoRA and explained the whole process in their insightful blogpost https://lnkd.in/g6jgfXyv. This is now integrated into Hugging Face ecosystem.

For an end-to-end example on FSDP+QLoRA, please refer https://lnkd.in/gT3yY-Rx.

For an end-to-end example on DeepSpeed Stage-3+QLoRA, please refer https://lnkd.in/gkt-xZRE.

With the PR https://lnkd.in/g5F348MN these changes are now upstreamed in https://lnkd.in/g5_MxYtY thanks to Wing Lian ! 🚀

Kudos to http://Answer.AI team,  Titus von Köller , Younes Belkada, Benjamin Bossan and Zachary Mueller for all the help without which this couldn't have been possible. 🤗

For efficient depthwise layer expansion akin to `passthrough` method of `mergekit` but without using additional memory and attaching LoRAs to it, refer to the details below! 🔥https://lnkd.in/ge95ztjA

Now DoRA is supported for Conv2D layers as well as bitsandbytes quantized layers ✨. For more details, please refer the below thread.
https://lnkd.in/gsJbuWPD

Now you can mix different LoRA adapters in a batch during inference which speeds-up the inference by avoiding computation of base model multiple times which would be the case for adaptive inference with batch_size=1! ⚡️.
Details below. https://lnkd.in/gD-pcX_B

LoftQ reduces quantization error by appropriately initializing the LoRA adapter weights. Normally, this is a two-step process. Benjamin Bossan
added new util `replace_lora_weights_loftq` for LoftQ to use it on the fly with bnb.

For more details, refer to the release notes. 📝
https://lnkd.in/gg7-AmHA. As always, make sure losses go down and be happy to watch your model train!
❤️
osanseviero clefourrier DmitryRyumin nbroad samusenps adamo1139 giux78 louisbrulenaudet sergey21000
🔥
osanseviero damerajee masterfury SicariusSicariiStuff giux78
🚀
osanseviero Norod78
😎
osanseviero
👍
osanseviero
🤝
osanseviero
🤯
osanseviero
👀
osanseviero
🧠
osanseviero
https://cdn-avatars.huggingface.co/v1/production/uploads/6569216f9c96f1a47bf45788/mCLqmAs4dOjKdxNQVAp1w.png
Sica Rius
SicariusSicariiStuff
user