text
‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 
‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 
new_line
new_line
text
1️⃣ Training Refactor
1️⃣ Training Refactor
new_line
text
Embedding models can now be trained using an extensive trainer with a lot of powerful features:
Embedding models can now be trained using an extensive trainer with a lot of powerful features:
new_line
text
- MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
- MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
new_line
text
- bf16 training support; loss logging
- bf16 training support; loss logging
new_line
text
- Evaluation datasets + evaluation loss
- Evaluation datasets + evaluation loss
new_line
text
- Improved callback support + an excellent Weights & Biases integration
- Improved callback support + an excellent Weights & Biases integration
new_line
text
- Gradient checkpointing, gradient accumulation
- Gradient checkpointing, gradient accumulation
new_line
text
- Improved model card generation
- Improved model card generation
new_line
text
- Resuming from a training checkpoint without performance loss
- Resuming from a training checkpoint without performance loss
new_line
text
- Hyperparameter Optimization
- Hyperparameter Optimization
new_line
text
and much more!
and much more!
new_line
text
Read my detailed blogpost to learn about the components that make up this new training approach: 
Read my detailed blogpost to learn about the components that make up this new training approach: 
link
https://huggingface.co/blog/train-sentence-transformers
https://huggingface.co/blog/train-sentence-transformers
new_line
new_line
text
2️⃣ Similarity Score
2️⃣ Similarity Score
new_line
text
Not sure how to compare embeddings? Don't worry, you can now use 
Not sure how to compare embeddings? Don't worry, you can now use 
inline_code
`model.similarity(embeddings1, embeddings2)`
model.similarity(embeddings1, embeddings2)
text
 and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!
 and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!
new_line
new_line
text
3️⃣ Additional Kwargs
3️⃣ Additional Kwargs
new_line
text
Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!
Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!
new_line
new_line
text
4️⃣ Hyperparameter Optimization
4️⃣ Hyperparameter Optimization
new_line
text
Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.
Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.
new_line
new_line
text
5️⃣ Dataset Release
5️⃣ Dataset Release
new_line
text
To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: 
To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: 
resource
https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
collection
sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
new_line
new_line
text
Full release notes: 
Full release notes: 
link
https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 

1️⃣ Training Refactor
Embedding models can now be trained using an extensive trainer with a lot of powerful features:
- MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
- bf16 training support; loss logging
- Evaluation datasets + evaluation loss
- Improved callback support + an excellent Weights & Biases integration
- Gradient checkpointing, gradient accumulation
- Improved model card generation
- Resuming from a training checkpoint without performance loss
- Hyperparameter Optimization
and much more!
Read my detailed blogpost to learn about the components that make up this new training approach: https://huggingface.co/blog/train-sentence-transformers

2️⃣ Similarity Score
Not sure how to compare embeddings? Don't worry, you can now use `model.similarity(embeddings1, embeddings2)` and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!

3️⃣ Additional Kwargs
Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!

4️⃣ Hyperparameter Optimization
Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.

5️⃣ Dataset Release
To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552

Full release notes: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
🔥
osanseviero alvarobartt lunarflu bourdoiscatie philschmid mattmdjaga DmitryRyumin not-lain gabrielmbmb cschroeder Ali-Khaled wonderinos urchade nickprock SamuelNthoroane Rybens vaclavkosar Abhinay45
❤️
louisbrulenaudet alvarobartt lunarflu bourdoiscatie philschmid gabrielmbmb urchade MexIvanov
🤗
lunarflu bourdoiscatie philschmid gabrielmbmb urchade
👍
trenkert