text
Google DeepMind introduces Gecko a new text embedding! Gecko uses a two-step process that leverages synthetic data generation and reranking.
Google DeepMind introduces Gecko a new text embedding! Gecko uses a two-step process that leverages synthetic data generation and reranking.
new_line
new_line
text
Keypoints:
Keypoints:
new_line
text
* Uses an LLM to generate diverse synthetic queries and tasks from web passages
* Uses an LLM to generate diverse synthetic queries and tasks from web passages
new_line
text
* Refines the data by retrieving candidate passages and relabeling positives/negatives using the same LLM
* Refines the data by retrieving candidate passages and relabeling positives/negatives using the same LLM
new_line
text
* Achieves very good results on the Massive Text Embedding Benchmark, where compact 256D Gecko outperforms 768D models.
* Achieves very good results on the Massive Text Embedding Benchmark, where compact 256D Gecko outperforms 768D models.
new_line
text
* 768D Gecko achieves state-of-the-art performance competing with models a lot larger larger.
* 768D Gecko achieves state-of-the-art performance competing with models a lot larger larger.
new_line
new_line
text
Paper: 
Paper: 
resource
https://huggingface.co/papers/2403.20327
paper
2403.20327
https://huggingface.co/papers/2403.20327
Gecko: Versatile Text Embeddings Distilled from Large Language Models (2403.20327)
new_line
text
More details in my blog:  
More details in my blog:  
link
https://huggingface.co/blog/vladbogo/gecko
https://huggingface.co/blog/vladbogo/gecko
new_line
new_line
text
Congrats to the authors for their work!
Congrats to the authors for their work!
Google DeepMind introduces Gecko a new text embedding! Gecko uses a two-step process that leverages synthetic data generation and reranking.

Keypoints:
* Uses an LLM to generate diverse synthetic queries and tasks from web passages
* Refines the data by retrieving candidate passages and relabeling positives/negatives using the same LLM
* Achieves very good results on the Massive Text Embedding Benchmark, where compact 256D Gecko outperforms 768D models.
* 768D Gecko achieves state-of-the-art performance competing with models a lot larger larger.

Paper: https://huggingface.co/papers/2403.20327
More details in my blog:  https://huggingface.co/blog/vladbogo/gecko

Congrats to the authors for their work!
❤️
samusenps clem monsoon-nlp beekay24 mohammedbriman lunarflu nikgr tatv047 super-cinnamon