text
Isn't it sad that VLMs don't have any inference parameters for the vision part? Well, MC-LLaVA now has two whole knobs you can use to make it find even the smallest details! I finally (almost) properly implemented multi-crop, and now you can control the number of crops and how many image tokens will be generated. The video shows how, by increasing the number of crops and tokens, my 3B model correctly identifies the 30x90 pixel logo in the 3200x3000 pixel image.
Isn't it sad that VLMs don't have any inference parameters for the vision part? Well, MC-LLaVA now has two whole knobs you can use to make it find even the smallest details! I finally (almost) properly implemented multi-crop, and now you can control the number of crops and how many image tokens will be generated. The video shows how, by increasing the number of crops and tokens, my 3B model correctly identifies the 30x90 pixel logo in the 3200x3000 pixel image.
new_line
text
Other notable updates:
Other notable updates:
new_line
text
- I use SigLIP from Transformers, so you don't need to install additional libraries.
- I use SigLIP from Transformers, so you don't need to install additional libraries.
new_line
text
- the model now supports auto classes, so you can create the model and processor with only two lines.
- the model now supports auto classes, so you can create the model and processor with only two lines.
new_line
text
- performance increased by 10%+ across all benchmarks.
- performance increased by 10%+ across all benchmarks.
new_line
new_line
text
The work is far from over, but it feels like good progress.
The work is far from over, but it feels like good progress.
new_line
new_line
text
The model on the hub: 
The model on the hub: 
resource
https://huggingface.co/visheratin/MC-LLaVA-3b
model
visheratin/MC-LLaVA-3b
https://huggingface.co/visheratin/MC-LLaVA-3b
new_line
text
You can try the model here: 
You can try the model here: 
resource
https://huggingface.co/spaces/visheratin/mc-llava-3b
space
visheratin/mc-llava-3b
https://huggingface.co/spaces/visheratin/mc-llava-3b
Isn't it sad that VLMs don't have any inference parameters for the vision part? Well, MC-LLaVA now has two whole knobs you can use to make it find even the smallest details! I finally (almost) properly implemented multi-crop, and now you can control the number of crops and how many image tokens will be generated. The video shows how, by increasing the number of crops and tokens, my 3B model correctly identifies the 30x90 pixel logo in the 3200x3000 pixel image.
Other notable updates:
- I use SigLIP from Transformers, so you don't need to install additional libraries.
- the model now supports auto classes, so you can create the model and processor with only two lines.
- performance increased by 10%+ across all benchmarks.

The work is far from over, but it feels like good progress.

The model on the hub: https://huggingface.co/visheratin/MC-LLaVA-3b
You can try the model here: https://huggingface.co/spaces/visheratin/mc-llava-3b
👍
samusenps osanseviero Sylvestre lusstta kinetical dillfrescott Azamat1k Alexandro14
❤️
samusenps clem lusstta okbottom dillfrescott