text
NVIDIA just dropped NVEagle 🦅
NVIDIA just dropped NVEagle 🦅
new_line
new_line
text
Super impressive vision language model that comes in 7B, 13B and 13B fine-tuned on chat 💬
Super impressive vision language model that comes in 7B, 13B and 13B fine-tuned on chat 💬
new_line
text
Model repositories: 
Model repositories: 
resource
https://huggingface.co/collections/merve/nveagle-66d0705108582d73bb235c26
collection
merve/nveagle-66d0705108582d73bb235c26
https://huggingface.co/collections/merve/nveagle-66d0705108582d73bb235c26
new_line
text
Try it: 
Try it: 
resource
https://huggingface.co/spaces/NVEagle/Eagle-X5-13B-Chat
space
NVEagle/Eagle-X5-13B-Chat
https://huggingface.co/spaces/NVEagle/Eagle-X5-13B-Chat
text
 💬 (works very well! 🤯)
 💬 (works very well! 🤯)
new_line
new_line
text
This model essentially explores having different experts (MoE) for image encoder part of vision language model. 
This model essentially explores having different experts (MoE) for image encoder part of vision language model. 
new_line
text
How? 🧐
How? 🧐
new_line
text
The authors concatenate the vision encoder output tokens together, and they apply "pre-alignment" essentially fine-tune experts with frozen text encoder. 
The authors concatenate the vision encoder output tokens together, and they apply "pre-alignment" essentially fine-tune experts with frozen text encoder. 
new_line
new_line
text
Then they freeze both experts and the decoder and just train the projection layer, and finally, they unfreeze everything for supervised fine-tuning ✨
Then they freeze both experts and the decoder and just train the projection layer, and finally, they unfreeze everything for supervised fine-tuning ✨
new_line
new_line
text
In the paper, they explore different fusion strategies and vision encoders, extending basic CLIP encoder, and figure out simply concatenating visual tokens works well.
In the paper, they explore different fusion strategies and vision encoders, extending basic CLIP encoder, and figure out simply concatenating visual tokens works well.
new_line
text
Rest of the architecture is quite similar to LLaVA. (see below the architecture) 
Rest of the architecture is quite similar to LLaVA. (see below the architecture) 
NVIDIA just dropped NVEagle 🦅

Super impressive vision language model that comes in 7B, 13B and 13B fine-tuned on chat 💬
Model repositories: https://huggingface.co/collections/merve/nveagle-66d0705108582d73bb235c26
Try it: https://huggingface.co/spaces/NVEagle/Eagle-X5-13B-Chat 💬 (works very well! 🤯)

This model essentially explores having different experts (MoE) for image encoder part of vision language model. 
How? 🧐
The authors concatenate the vision encoder output tokens together, and they apply "pre-alignment" essentially fine-tune experts with frozen text encoder. 

Then they freeze both experts and the decoder and just train the projection layer, and finally, they unfreeze everything for supervised fine-tuning ✨

In the paper, they explore different fusion strategies and vision encoders, extending basic CLIP encoder, and figure out simply concatenating visual tokens works well.
Rest of the architecture is quite similar to LLaVA. (see below the architecture) 
👍
sagar007 Kukedlc John6666 khanhnamle1994 maximuspowers alielfilali01 Arakinas darkzbaron damerajee Tahahah andito noobmldude
🚀
damerajee Tahahah louisbrulenaudet andito