|
--- |
|
license: mit |
|
language: |
|
- en |
|
--- |
|
|
|
# Model Details |
|
Agglomerative Token Clustering (ATC), a novel hierarchical hard-merging based token reduction method. |
|
ATC has been demonstrated to outperform all prior state-of-the-art token reduction methods across image classification, image synthesis, and object detection & segementation tasks. |
|
|
|
We provide DeiT checkpoints (Tiny, Small, and Base) at four reduction rates (0.9, 0.7, 0.5, and 0.25) for four classification datasets: ImageNet-1K, NABirds, COCO 2014, and NUS-WIDE. |
|
We also provide ViT-Adapter checkpoints (Tiny and Small) for the COCO 2017 object detection and segementation task, for both the ATC and ToMe token reduction methods. |
|
|
|
|
|
### Model Description |
|
- **Developed by:** Joakim Bruslund Haurum, Sergio Escalera, Graham W. Taylor, and Thomas B. Moeslund |
|
- **Model type:** Vision Transformers |
|
- **License:** MIT License |
|
|
|
### More Resources |
|
- **Repository:** [https://github.com/JoakimHaurum/ATC](https://github.com/JoakimHaurum/ATC) |
|
- **Paper:** [https://arxiv.org/abs/2409.11923](https://arxiv.org/abs/2409.11923) |
|
- **Project Page:** [https://vap.aau.dk/atc](https://vap.aau.dk/atc) |
|
- **HuggingFace Collection:** [https://huggingface.co/collections/joakimbh/agglomerative-token-clustering-66e94dfb313e85ec97590fe4](https://huggingface.co/collections/joakimbh/agglomerative-token-clustering-66e94dfb313e85ec97590fe4) |
|
|
|
|
|
## Model Zoo |
|
**Note: This repository does not host any checkpoints but contains links to all the model repositories. Each token reduction method repository contains the checkpoints for the four considered keep rates.** |
|
|
|
Image Classification Models: |
|
| Model Name | Dataset | Linkage Function | Weights | |
|
|:-----------|:--------:|:-----------:|:--------:| |
|
| atc_base-im1k-average | ImageNet-1K | Average | [link](https://huggingface.co/joakimbh/atc_base-im1k-average) | |
|
| atc_base-im1k-single | ImageNet-1K | Single | [link](https://huggingface.co/joakimbh/atc_base-im1k-single) | |
|
| atc_base-im1k-complete | ImageNet-1K | Complete | [link](https://huggingface.co/joakimbh/atc_base-im1k-complete) | |
|
| atc_small-im1k-average | ImageNet-1K | Average | [link](https://huggingface.co/joakimbh/atc_small-im1k-average) | |
|
| atc_small-im1k-single | ImageNet-1K | Single | [link](https://huggingface.co/joakimbh/atc_small-im1k-single) | |
|
| atc_small-im1k-complete | ImageNet-1K | Complete | [link](https://huggingface.co/joakimbh/atc_small-im1k-complete) | |
|
| atc_tiny-im1k-average | ImageNet-1K | Average | [link](https://huggingface.co/joakimbh/atc_tiny-im1k-average) | |
|
| atc_tiny-im1k-single | ImageNet-1K | Single | [link](https://huggingface.co/joakimbh/atc_tiny-im1k-single) | |
|
| atc_tiny-im1k-complete | ImageNet-1K | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-im1k-complete) | |
|
<br> |
|
| atc_base-nab-average | NABirds | Average | [link](https://huggingface.co/joakimbh/atc_base-nab-average) | |
|
| atc_base-nab-single | NABirds | Single | [link](https://huggingface.co/joakimbh/atc_base-nab-single) | |
|
| atc_base-nab-complete | NABirds | Complete | [link](https://huggingface.co/joakimbh/atc_base-nab-complete) | |
|
| atc_small-nab-average | NABirds | Average | [link](https://huggingface.co/joakimbh/atc_small-nab-average) | |
|
| atc_small-nab-single | NABirds | Single | [link](https://huggingface.co/joakimbh/atc_small-nab-single) | |
|
| atc_small-nab-complete | NABirds | Complete | [link](https://huggingface.co/joakimbh/atc_small-nab-complete) | |
|
| atc_tiny-nab-average | NABirds | Average | [link](https://huggingface.co/joakimbh/atc_tiny-nab-average) | |
|
| atc_tiny-nab-single | NABirds | Single | [link](https://huggingface.co/joakimbh/atc_tiny-nab-single) | |
|
| atc_tiny-nab-complete | NABirds | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-nab-complete) | |
|
<br> |
|
| atc_base-coco-average | COCO 2014 | Average | [link](https://huggingface.co/joakimbh/atc_base-coco-average) | |
|
| atc_base-coco-single | COCO 2014 | Single | [link](https://huggingface.co/joakimbh/atc_base-coco-single) | |
|
| atc_base-coco-complete | COCO 2014 | Complete | [link](https://huggingface.co/joakimbh/atc_base-coco-complete) | |
|
| atc_small-coco-average | COCO 2014 | Average | [link](https://huggingface.co/joakimbh/atc_small-coco-average) | |
|
| atc_small-coco-single | COCO 2014 | Single | [link](https://huggingface.co/joakimbh/atc_small-coco-single) | |
|
| atc_small-coco-complete | COCO 2014 | Complete | [link](https://huggingface.co/joakimbh/atc_small-coco-complete) | |
|
| atc_tiny-coco-average | COCO 2014 | Average | [link](https://huggingface.co/joakimbh/atc_tiny-coco-average) | |
|
| atc_tiny-coco-single | COCO 2014 | Single | [link](https://huggingface.co/joakimbh/atc_tiny-coco-single) | |
|
| atc_tiny-coco-complete | COCO 2014 | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-coco-complete) | |
|
<br> |
|
| atc_base-nus-average | NUS-WIDE | Average | [link](https://huggingface.co/joakimbh/atc_base-nus-average) | |
|
| atc_base-nus-single | NUS-WIDE | Single | [link](https://huggingface.co/joakimbh/atc_base-nus-single) | |
|
| atc_base-nus-complete | NUS-WIDE | Complete | [link](https://huggingface.co/joakimbh/atc_base-nus-complete) | |
|
| atc_small-nus-average | NUS-WIDE | Average | [link](https://huggingface.co/joakimbh/atc_small-nus-average) | |
|
| atc_small-nus-single | NUS-WIDE | Single | [link](https://huggingface.co/joakimbh/atc_small-nus-single) | |
|
| atc_small-nus-complete | NUS-WIDE | Complete | [link](https://huggingface.co/joakimbh/atc_small-nus-complete) | |
|
| atc_tiny-nus-average| NUS-WIDE | Average | [link](https://huggingface.co/joakimbh/atc_tiny-nus-average) | |
|
| atc_tiny-nus-single | NUS-WIDE | Single | [link](https://huggingface.co/joakimbh/atc_tiny-nus-single) | |
|
| atc_tiny-nus-complete | NUS-WIDE | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-nus-complete) | |
|
<br> |
|
|
|
Object Detection and Segmentation Models: |
|
| Model Name | Dataset | Keep Rate | Weights | |
|
|:-----------|:--------:|:-----------:|:--------:| |
|
| atc_small-coco17-average | COCO 2017 | Average | [link](https://huggingface.co/joakimbh/atc_small-coco17-average) | |
|
| atc_small-coco17-single | COCO 2017 | Single | [link](https://huggingface.co/joakimbh/atc_small-coco17-single) | |
|
| atc_small-coco17-complete | COCO 2017 | Complete | [link](https://huggingface.co/joakimbh/atc_small-coco17-complete) | |
|
| tome_small-coco17 | COCO 2017 | - | [link](https://huggingface.co/joakimbh/tome_small-coco17) | |
|
<br> |
|
| atc_tiny-coco17-average | COCO 2017 | Average | [link](https://huggingface.co/joakimbh/atc_tiny-coco17-average) | |
|
| atc_tiny-coco17-single | COCO 2017 | Single | [link](https://huggingface.co/joakimbh/atc_tiny-coco17-single) | |
|
| atc_tiny-coco17-complete | COCO 2017 | Complete | [link](https://huggingface.co/joakimbh/atc_tiny-coco17-complete) | |
|
| tome_tiny-coco17 | COCO 2017 | - | [link](https://huggingface.co/joakimbh/tome_tiny-coco17) | |
|
<br> |
