deepseek-ai/DeepSeek-V3
Updated
•
74.1k
•
1.41k
Syahmi Azhar
prsyahmi
AI & ML interests
None yet
Recent Activity
liked
a model
5 days ago
deepseek-ai/DeepSeek-V3
reacted
to
MoritzLaurer's
post
with 👍
17 days ago
Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification!
This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021 :D
Congrats @answerdotai, @LightOnIO and collaborators like @tomaarsen !
Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
Organizations
None yet
prsyahmi's activity
liked
a
model
5 days ago
reacted to
ginipick's
post with 👍
17 days ago
Post
4293
🌟 Digital Odyssey: AI Image & Video Generation Platform 🎨
Welcome to our all-in-one AI platform for image and video generation! 🚀
✨ Key Features
🎨 High-quality image generation from text
🎥 Video creation from still images
🌐 Multi-language support with automatic translation
🛠️ Advanced customization options
💫 Unique Advantages
⚡ Fast and accurate results using FLUX.1-dev and Hyper-SD models
🔒 Robust content safety filtering system
🎯 Intuitive user interface
🛠️ Extended toolkit including image upscaling and logo generation
🎮 How to Use
Enter your image or video description
Adjust settings as needed
Click generate
Save and share your results automatically
🔧 Tech Stack
FluxPipeline
Gradio
PyTorch
OpenCV
link: ginigen/Dokdo
Turn your imagination into reality with AI! ✨
#AI #ImageGeneration #VideoGeneration #MachineLearning #CreativeTech
Welcome to our all-in-one AI platform for image and video generation! 🚀
✨ Key Features
🎨 High-quality image generation from text
🎥 Video creation from still images
🌐 Multi-language support with automatic translation
🛠️ Advanced customization options
💫 Unique Advantages
⚡ Fast and accurate results using FLUX.1-dev and Hyper-SD models
🔒 Robust content safety filtering system
🎯 Intuitive user interface
🛠️ Extended toolkit including image upscaling and logo generation
🎮 How to Use
Enter your image or video description
Adjust settings as needed
Click generate
Save and share your results automatically
🔧 Tech Stack
FluxPipeline
Gradio
PyTorch
OpenCV
link: ginigen/Dokdo
Turn your imagination into reality with AI! ✨
#AI #ImageGeneration #VideoGeneration #MachineLearning #CreativeTech
- 7 replies
reacted to
MoritzLaurer's
post with 👍
17 days ago
Post
2584
Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification!
This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021 :D
Congrats @answerdotai , @LightOnIO and collaborators like @tomaarsen !
Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021 :D
Congrats @answerdotai , @LightOnIO and collaborators like @tomaarsen !
Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
- 3 replies
liked
a
model
19 days ago
liked
3
models
about 2 months ago
liked
a
model
2 months ago
liked
a
Space
2 months ago
liked
a
model
2 months ago
liked
a
model
3 months ago
liked
a
model
5 months ago
liked
a
model
6 months ago
liked
2
models
7 months ago
reacted to
singhsidhukuldeep's
post with 🤗
8 months ago
Post
1457
You are all happy 😊 that
@meta-llama
released Llama 3.
Then you are sad 😔 that it only has a context length of 8k.
Then you are happy 😄 that you can just scale llama-3 PoSE to 96k without training, only needing to modify max_position_embeddings and rope_theta.
But then you are sad 😢 it only improves the model's long-context retrieval performance (i.e., finding needles) while hardly improving its long-context utilization capability (doing QA and summarization).
But then you are happy 😁 that the
@GradientsTechnologies community has released the long-context Llama-3-8B-Instruct-262K with long context (262k-1M+).
Now we have another paper "Extending Llama-3's Context Ten-Fold Overnight" 📜.
The context length of Llama-3-8B-Instruct is extended from 8K to 80K using QLoRA fine-tuning⚙️.
The training cycle is highly efficient, taking "only" 😂 8 hours on a single 8xA800 (80G) GPU machine.
The model also preserves its original capability over short contexts. ✁
The dramatic context extension is mainly attributed to merely 3.5K synthetic training samples generated by GPT-4.📊
The paper suggests that the context length could be extended far beyond 80K with more computation resources (😅 GPU-poor).
The team plans to publicly release all resources, including data, model, data generation pipeline, and training code, to facilitate future research from the ❤️ community.
Paper: https://arxiv.org/abs/2404.19553
This is where we are... until next time... 🌟
Extending Llama-3's Context Ten-Fold Overnight (2404.19553)
Then you are sad 😔 that it only has a context length of 8k.
Then you are happy 😄 that you can just scale llama-3 PoSE to 96k without training, only needing to modify max_position_embeddings and rope_theta.
But then you are sad 😢 it only improves the model's long-context retrieval performance (i.e., finding needles) while hardly improving its long-context utilization capability (doing QA and summarization).
But then you are happy 😁 that the
@GradientsTechnologies community has released the long-context Llama-3-8B-Instruct-262K with long context (262k-1M+).
Now we have another paper "Extending Llama-3's Context Ten-Fold Overnight" 📜.
The context length of Llama-3-8B-Instruct is extended from 8K to 80K using QLoRA fine-tuning⚙️.
The training cycle is highly efficient, taking "only" 😂 8 hours on a single 8xA800 (80G) GPU machine.
The model also preserves its original capability over short contexts. ✁
The dramatic context extension is mainly attributed to merely 3.5K synthetic training samples generated by GPT-4.📊
The paper suggests that the context length could be extended far beyond 80K with more computation resources (😅 GPU-poor).
The team plans to publicly release all resources, including data, model, data generation pipeline, and training code, to facilitate future research from the ❤️ community.
Paper: https://arxiv.org/abs/2404.19553
This is where we are... until next time... 🌟
Extending Llama-3's Context Ten-Fold Overnight (2404.19553)
liked
a
Space
8 months ago
liked
a
model
8 months ago
upvoted
a
paper
9 months ago
liked
a
Space
9 months ago