Image Feature Extraction
Transformers
Safetensors
dinov2
Inference Endpoints
fepegar commited on
Commit
e8051ea
·
verified ·
1 Parent(s): f8c75fd

Fix usage snippet

Browse files
Files changed (1) hide show
  1. README.md +9 -7
README.md CHANGED
@@ -13,7 +13,7 @@ license: mit
13
 
14
  RAD-DINO is a vision transformer model trained to encode chest X-rays using the self-supervised learning method [DINOv2](https://openreview.net/forum?id=a68SUt6zFt).
15
 
16
- RAD-DINO is described in detail in [RAD-DINO: Exploring Scalable Medical Image Encoders Beyond Text Supervision (Pérez-García, Sharma, Bond-Taylor et al., 2024)](https://arxiv.org/abs/2401.10815).
17
 
18
  - **Developed by:** Microsoft Health Futures
19
  - **Model type:** Vision transformer
@@ -46,7 +46,7 @@ Fine-tuning RAD-DINO is typically not necessary to obtain good performance in do
46
 
47
  <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
48
 
49
- ## Bias, risks, and limitations
50
 
51
  <!-- This section is meant to convey both technical and sociotechnical limitations. -->
52
 
@@ -74,15 +74,17 @@ Underlying biases of the training datasets may not be well characterised.
74
  ...
75
  >>> # Download the model
76
  >>> repo = "microsoft/rad-dino"
77
- >>> model = AutoModel.from_pretrained(repo).cuda()
78
 
79
  >>> # The processor takes a PIL image, performs resizing, center-cropping, and
80
  >>> # intensity normalization using stats from MIMIC-CXR, and returns a
81
  >>> # dictionary with a PyTorch tensor ready for the encoder
82
- >>> processor = AutoProcessor.from_pretrained(repo)
83
  >>>
84
  >>> # Download and preprocess a chest X-ray
85
  >>> image = download_sample_image()
 
 
86
  >>> inputs = processor(images=image, return_tensors="pt")
87
  >>>
88
  >>> # Encode the image!
@@ -90,8 +92,8 @@ Underlying biases of the training datasets may not be well characterised.
90
  >>> outputs = model(**inputs)
91
  >>>
92
  >>> # Look at the CLS embeddings
93
- >>> cls_embeddings = outputs.pooler_output.shape
94
- >>> cls_embeddings # (batch_size, num_channels)
95
  torch.Size([1, 768])
96
  >>>
97
  >>> # Look at the patch embeddings (needs `pip install einops`)
@@ -220,4 +222,4 @@ We used [SimpleITK](https://simpleitk.org/) and [Pydicom](https://pydicom.github
220
 
221
  ## Model card contact
222
 
223
- Fernando Pérez-García ([`[email protected]`](mailto:[email protected])).
 
13
 
14
  RAD-DINO is a vision transformer model trained to encode chest X-rays using the self-supervised learning method [DINOv2](https://openreview.net/forum?id=a68SUt6zFt).
15
 
16
+ RAD-DINO is described in detail in [RAD-DINO: Exploring Scalable Medical Image Encoders Beyond Text Supervision (F. Pérez-García, H. Sharma, S. Bond-Taylor, et al., 2024)](https://arxiv.org/abs/2401.10815).
17
 
18
  - **Developed by:** Microsoft Health Futures
19
  - **Model type:** Vision transformer
 
46
 
47
  <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
48
 
49
+ ## Biases, risks, and limitations
50
 
51
  <!-- This section is meant to convey both technical and sociotechnical limitations. -->
52
 
 
74
  ...
75
  >>> # Download the model
76
  >>> repo = "microsoft/rad-dino"
77
+ >>> model = AutoModel.from_pretrained(repo)
78
 
79
  >>> # The processor takes a PIL image, performs resizing, center-cropping, and
80
  >>> # intensity normalization using stats from MIMIC-CXR, and returns a
81
  >>> # dictionary with a PyTorch tensor ready for the encoder
82
+ >>> processor = AutoImageProcessor.from_pretrained(repo)
83
  >>>
84
  >>> # Download and preprocess a chest X-ray
85
  >>> image = download_sample_image()
86
+ >>> image.size # (width, height)
87
+ (2765, 2505)
88
  >>> inputs = processor(images=image, return_tensors="pt")
89
  >>>
90
  >>> # Encode the image!
 
92
  >>> outputs = model(**inputs)
93
  >>>
94
  >>> # Look at the CLS embeddings
95
+ >>> cls_embeddings = outputs.pooler_output
96
+ >>> cls_embeddings.shape # (batch_size, num_channels)
97
  torch.Size([1, 768])
98
  >>>
99
  >>> # Look at the patch embeddings (needs `pip install einops`)
 
222
 
223
  ## Model card contact
224
 
225
+ Fernando Pérez-García ([`[email protected]`](mailto:[email protected])).