text
🪟32k-context BERT for embedding and RAG on long corpus
🪟32k-context BERT for embedding and RAG on long corpus
new_line
new_line
text
Monarch Mixer is a new architecture to enable long context BERT for large corpus and can be fine-tuned for large context retrieval.
Monarch Mixer is a new architecture to enable long context BERT for large corpus and can be fine-tuned for large context retrieval.
new_line
new_line
text
Quite interesting and important as BERT is still the most used LLM in production for "old school" tasks like classification, NER, embeddings, but is also a key component for RAG.
Quite interesting and important as BERT is still the most used LLM in production for "old school" tasks like classification, NER, embeddings, but is also a key component for RAG.
new_line
new_line
text
Paper: 
Paper: 
link
https://arxiv.org/abs/2310.12109
https://arxiv.org/abs/2310.12109
new_line
text
Blog: 
Blog: 
link
https://hazyresearch.stanford.edu/blog/2024-01-11-m2-bert-retrieval
https://hazyresearch.stanford.edu/blog/2024-01-11-m2-bert-retrieval
new_line
text
GitHub: 
GitHub: 
link
https://github.com/HazyResearch/m2
https://github.com/HazyResearch/m2
🪟32k-context BERT for embedding and RAG on long corpus

Monarch Mixer is a new architecture to enable long context BERT for large corpus and can be fine-tuned for large context retrieval.

Quite interesting and important as BERT is still the most used LLM in production for "old school" tasks like classification, NER, embeddings, but is also a key component for RAG.

Paper: https://arxiv.org/abs/2310.12109
Blog: https://hazyresearch.stanford.edu/blog/2024-01-11-m2-bert-retrieval
GitHub: https://github.com/HazyResearch/m2
❤️
s3nh abidlabs osanseviero samusenps DRXD1000 Shir02021 Tonic clem yash-srivastava19 abdullah d0rj sosoai
👍
roseking liyucheng
https://cdn-avatars.huggingface.co/v1/production/uploads/62a3bb1cd0d8c2c2169f0b88/eT2TS0IlQbZtz-F_zHLz9.jpeg
Joseph [open/acc] Pollack
Tonic
user