text
We have recently merged Video-LLaVA to transformers! 🤗🎞️
We have recently merged Video-LLaVA to transformers! 🤗🎞️
new_line
text
What makes this model different?
What makes this model different?
new_line
new_line
text
Demo: 
Demo: 
resource
https://huggingface.co/spaces/llava-hf/video-llava
space
llava-hf/video-llava
https://huggingface.co/spaces/llava-hf/video-llava
new_line
text
Model: 
Model: 
resource
https://huggingface.co/LanguageBind/Video-LLaVA-7B-hf
model
LanguageBind/Video-LLaVA-7B-hf
https://huggingface.co/LanguageBind/Video-LLaVA-7B-hf
new_line
new_line
text
Compared to other models that take image and video input and either project them separately or downsampling video and projecting selected frames, Video-LLaVA is converting images and videos to unified representation and project them using a shared projection layer.
Compared to other models that take image and video input and either project them separately or downsampling video and projecting selected frames, Video-LLaVA is converting images and videos to unified representation and project them using a shared projection layer.
new_line
new_line
text
It uses Vicuna 1.5 as the language model and LanguageBind's own encoders that's based on OpenCLIP, these encoders project the modalities to an unified representation before passing to projection layer.
It uses Vicuna 1.5 as the language model and LanguageBind's own encoders that's based on OpenCLIP, these encoders project the modalities to an unified representation before passing to projection layer.
new_line
new_line
new_line
text
I feel like one of the coolest features of this model is the joint understanding which is also introduced recently with many models
I feel like one of the coolest features of this model is the joint understanding which is also introduced recently with many models
new_line
new_line
text
It's a relatively older model but ahead of it's time and works very well! Which means, e.g. you can pass model an image of a cat and a video of a cat and ask questions like whether the cat in the image exists in video or not 🤩
It's a relatively older model but ahead of it's time and works very well! Which means, e.g. you can pass model an image of a cat and a video of a cat and ask questions like whether the cat in the image exists in video or not 🤩
new_line
We have recently merged Video-LLaVA to transformers! 🤗🎞️
What makes this model different?

Demo: https://huggingface.co/spaces/llava-hf/video-llava
Model: https://huggingface.co/LanguageBind/Video-LLaVA-7B-hf

Compared to other models that take image and video input and either project them separately or downsampling video and projecting selected frames, Video-LLaVA is converting images and videos to unified representation and project them using a shared projection layer.

It uses Vicuna 1.5 as the language model and LanguageBind's own encoders that's based on OpenCLIP, these encoders project the modalities to an unified representation before passing to projection layer.


I feel like one of the coolest features of this model is the joint understanding which is also introduced recently with many models

It's a relatively older model but ahead of it's time and works very well! Which means, e.g. you can pass model an image of a cat and a video of a cat and ask questions like whether the cat in the image exists in video or not 🤩

🔥
adorkin orkut osanseviero Andyrasika SizorCloud
🚀
prithivMLmods Andyrasika NHLOCAL
❤️
Tonic