text
🎉  We are thrilled to share our work on model merging. We proposed a new approach, Della-merging, which combines expert models from various domains into a single, versatile model. Della employs a magnitude-based sampling approach to eliminate redundant delta parameters, reducing interference when merging homologous models (those fine-tuned from the same backbone).
🎉  We are thrilled to share our work on model merging. We proposed a new approach, Della-merging, which combines expert models from various domains into a single, versatile model. Della employs a magnitude-based sampling approach to eliminate redundant delta parameters, reducing interference when merging homologous models (those fine-tuned from the same backbone).
new_line
new_line
text
Della outperforms existing homologous model merging techniques such as DARE and TIES. Across three expert models (LM, Math, Code) and their corresponding benchmark datasets (AlpacaEval, GSM8K, MBPP), Della achieves an improvement of 3.6 points over TIES and 1.2 points over DARE.
Della outperforms existing homologous model merging techniques such as DARE and TIES. Across three expert models (LM, Math, Code) and their corresponding benchmark datasets (AlpacaEval, GSM8K, MBPP), Della achieves an improvement of 3.6 points over TIES and 1.2 points over DARE.
new_line
new_line
text
Paper: 
Paper: 
resource
https://huggingface.co/papers/2406.11617
paper
2406.11617
https://huggingface.co/papers/2406.11617
DELLA-Merging: Reducing Interference in Model Merging through
  Magnitude-Based Sampling (2406.11617)
new_line
text
Github: 
Github: 
link
https://github.com/declare-lab/della
https://github.com/declare-lab/della
new_line
new_line
mention
@soujanyaporia
soujanyaporia
text
mention
@Tej3
Tej3
text
🎉  We are thrilled to share our work on model merging. We proposed a new approach, Della-merging, which combines expert models from various domains into a single, versatile model. Della employs a magnitude-based sampling approach to eliminate redundant delta parameters, reducing interference when merging homologous models (those fine-tuned from the same backbone).

Della outperforms existing homologous model merging techniques such as DARE and TIES. Across three expert models (LM, Math, Code) and their corresponding benchmark datasets (AlpacaEval, GSM8K, MBPP), Della achieves an improvement of 3.6 points over TIES and 1.2 points over DARE.

Paper: https://huggingface.co/papers/2406.11617
Github: https://github.com/declare-lab/della

@soujanyaporia @Tej3 
🤯
YaTharThShaRma999 John6666 soujanyaporia RishabhBhardwaj
👍
dillfrescott QizhiPei
👀
louisbrulenaudet
https://cdn-avatars.huggingface.co/v1/production/uploads/65d883893a52cd9bcd8ab7cf/tRsCJlHNZo1D02kBTmfy9.jpeg
leroy Samuel Dyer
LeroyDyer
user
/avatars/48312867ac2c05d589869972f90881b9.svg
Pala Tej Deep
Tej3
user