
# Model card for vit_small_patch8_224.lunit_dino

A Vision Transformer (ViT) image classification model. \
Trained on 33M histology patches from various pathology datasets.

![](https://github.com/lunit-io/benchmark-ssl-pathology/raw/main/assets/ssl_teaser.png)

## Model Details

- **Model Type:** Feature backbone
- **SSL Method:** DINO
- **Model Stats:**
  - Params (M): 21.7
  - Image sizes: 224 × 224 x 3
- **Papers:**
  - Benchmarking Self-Supervised Learning on Diverse Pathology Datasets: https://arxiv.org/abs/2212.04690
- **Datasets:**
  -  BACH
  -  CRC
  -  MHIST
  -  PatchCamelyon
  -  CoNSeP
- **Original:** https://github.com/lunit-io/benchmark-ssl-pathology
- **License:** [lunit-non-commercial](https://github.com/lunit-io/benchmark-ssl-pathology/blob/main/LICENSE)

## Model Usage

### Image Embeddings
```python
from urllib.request import urlopen
from PIL import Image
import timm

# get example histology image
img = Image.open(
  urlopen(
    "https://github.com/owkin/HistoSSLscaling/raw/main/assets/example.tif"
  )
)

# load model from the hub
model = timm.create_model(
  model_name="hf-hub:1aurent/vit_small_patch8_224.lunit_dino",
  pretrained=True,
).eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor
```

## Citation
```bibtex
@inproceedings{kang2022benchmarking,
  author    = {Kang, Mingu and Song, Heon and Park, Seonwook and Yoo, Donggeun and Pereira, Sérgio},
  title     = {Benchmarking Self-Supervised Learning on Diverse Pathology Datasets},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2023},
  pages     = {3344-3354}
}
```
{"license": "other", "library_name": "timm", "tags": ["image-classification", "timm"], "datasets": ["1aurent/BACH", "1aurent/NCT-CRC-HE", "1aurent/PatchCamelyon"], "license_name": "lunit-non-commercial", "license_link": "https://github.com/lunit-io/benchmark-ssl-pathology/blob/main/LICENSE", "pipeline_tag": "image-classification"}
timm safetensors image-classification dataset:1aurent/BACH dataset:1aurent/NCT-CRC-HE dataset:1aurent/PatchCamelyon arxiv:2212.04690 license:other region:us