|
--- |
|
license: other |
|
license_name: nvclv1 |
|
license_link: LICENSE |
|
datasets: |
|
- ILSVRC/imagenet-1k |
|
pipeline_tag: image-classification |
|
--- |
|
|
|
|
|
[**MambaVision: A Hybrid Mamba-Transformer Vision Backbone**](https://arxiv.org/abs/2407.08083). |
|
|
|
### Model Overview |
|
|
|
We introduce a novel mixer block by creating a symmetric path without SSM to enhance the modeling of global context. MambaVision has a hierarchical architecture that employs both self-attention and mixer blocks. |
|
|
|
|
|
### Model Performance |
|
|
|
MambaVision demonstrates a strong performance by achieving a new SOTA Pareto-front in |
|
terms of Top-1 accuracy and throughput. |
|
|
|
<p align="center"> |
|
<img src="https://github.com/NVlabs/MambaVision/assets/26806394/79dcf841-3966-4b77-883d-76cd5e1d4320" width=42% height=42% |
|
class="center"> |
|
</p> |
|
|
|
|
|
### Model Usage |
|
|
|
You must first login into HuggingFace to pull the model: |
|
|
|
```Bash |
|
huggingface-cli login |
|
``` |
|
|
|
The model can be simply used according to: |
|
|
|
```Python |
|
access_token = "<YOUR ACCESS TOKEN" |
|
model = AutoModel.from_pretrained("nvidia/MambaVision-S-1K", trust_remote_code=True) |
|
``` |
|
|
|
|
|
### License: |
|
|
|
[NVIDIA Source Code License-NC](https://huggingface.co/nvidia/MambaVision-S-1K/blob/main/LICENSE) |
|
|
|
|
