Hugging Face's logo
Selim Özten's picture
1 43 209

Selim Özten PRO

comarproject
meguzn's profile picture 21world's profile picture atahanuz's profile picture
·
https://www.inpocket.ai

AI & ML interests

None yet

Recent Activity

liked a dataset about 3 hours ago
saillab/alpaca-turkish-cleaned
liked a model 16 days ago
answerdotai/ModernBERT-base
reacted to m-ric's post with 🔥 16 days ago
After 6 years, BERT, the workhorse of encoder models, finally gets a replacement: 𝗪𝗲𝗹𝗰𝗼𝗺𝗲 𝗠𝗼𝗱𝗲𝗿𝗻𝗕𝗘𝗥𝗧! 🤗 We talk a lot about ✨Generative AI✨, meaning "Decoder version of the Transformers architecture", but this is only one of the ways to build LLMs: encoder models, that turn a sentence in a vector, are maybe even more widely used in industry than generative models. The workhorse for this category has been BERT since its release in 2018 (that's prehistory for LLMs). It's not a fancy 100B parameters supermodel (just a few hundred millions), but it's an excellent workhorse, kind of a Honda Civic for LLMs. Many applications use BERT-family models - the top models in this category cumulate millions of downloads on the Hub. ➡️ Now a collaboration between Answer.AI and LightOn just introduced BERT's replacement: ModernBERT. 𝗧𝗟;𝗗𝗥: 🏛️ Architecture changes: ⇒ First, standard modernizations: - Rotary positional embeddings (RoPE) - Replace GeLU with GeGLU, - Use Flash Attention 2 ✨ The team also introduced innovative techniques like alternating attention instead of full attention, and sequence packing to get rid of padding overhead. 🥇 As a result, the model tops the game of encoder models: It beats previous standard DeBERTaV3 for 1/5th the memory footprint, and runs 4x faster! Read the blog post 👉 https://huggingface.co/blog/modernbert
View all activity

Organizations

inpocket.ai's profile picture AI Starter Pack's profile picture

comarproject's activity

liked a dataset about 3 hours ago
liked a model 16 days ago
liked 2 models 19 days ago
liked a Space 21 days ago
liked a model 26 days ago
liked a Space 28 days ago
liked a dataset 28 days ago
liked 3 models 28 days ago
liked a Space 28 days ago
liked a dataset 29 days ago
liked a Space about 1 month ago
liked 4 models about 1 month ago
liked a Space about 1 month ago
liked a model about 1 month ago