text
NVIDIA just dropped a gigantic multimodal model called NVLM 72B 🦖 
NVIDIA just dropped a gigantic multimodal model called NVLM 72B 🦖 
new_line
resource
https://huggingface.co/nvidia/NVLM-D-72B
model
nvidia/NVLM-D-72B
https://huggingface.co/nvidia/NVLM-D-72B
new_line
text
Paper page 
Paper page 
resource
https://huggingface.co/papers/2409.11402
paper
2409.11402
https://huggingface.co/papers/2409.11402
NVLM: Open Frontier-Class Multimodal LLMs (2409.11402)
new_line
new_line
text
The paper contains many ablation studies on various ways to use the LLM backbone 👇🏻
The paper contains many ablation studies on various ways to use the LLM backbone 👇🏻
new_line
new_line
text
🦩 Flamingo-like cross-attention (NVLM-X)
🦩 Flamingo-like cross-attention (NVLM-X)
new_line
text
🌋 Llava-like concatenation of image and text embeddings to a decoder-only model (NVLM-D)
🌋 Llava-like concatenation of image and text embeddings to a decoder-only model (NVLM-D)
new_line
text
✨ a hybrid architecture (NVLM-H)
✨ a hybrid architecture (NVLM-H)
new_line
new_line
text
Checking evaluations, NVLM-D and NVLM-H are best or second best compared to other models 👏
Checking evaluations, NVLM-D and NVLM-H are best or second best compared to other models 👏
new_line
new_line
text
The released model is NVLM-D based on Qwen-2 Instruct, aligned with InternViT-6B using a huge mixture of different datasets
The released model is NVLM-D based on Qwen-2 Instruct, aligned with InternViT-6B using a huge mixture of different datasets
new_line
new_line
text
You can easily use this model by loading it through transformers' AutoModel 😍
You can easily use this model by loading it through transformers' AutoModel 😍
NVIDIA just dropped a gigantic multimodal model called NVLM 72B 🦖 
https://huggingface.co/nvidia/NVLM-D-72B
Paper page https://huggingface.co/papers/2409.11402

The paper contains many ablation studies on various ways to use the LLM backbone 👇🏻

🦩 Flamingo-like cross-attention (NVLM-X)
🌋 Llava-like concatenation of image and text embeddings to a decoder-only model (NVLM-D)
✨ a hybrid architecture (NVLM-H)

Checking evaluations, NVLM-D and NVLM-H are best or second best compared to other models 👏

The released model is NVLM-D based on Qwen-2 Instruct, aligned with InternViT-6B using a huge mixture of different datasets

You can easily use this model by loading it through transformers' AutoModel 😍
🔥
louisbrulenaudet iojvsuynv KingNish DmitryRyumin namelessai AtAndDev John6666 garrethlee osanseviero djuna gizemsarsinlar