Fix usage snippet
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ license: mit
|
|
13 |
|
14 |
RAD-DINO is a vision transformer model trained to encode chest X-rays using the self-supervised learning method [DINOv2](https://openreview.net/forum?id=a68SUt6zFt).
|
15 |
|
16 |
-
RAD-DINO is described in detail in [RAD-DINO: Exploring Scalable Medical Image Encoders Beyond Text Supervision (Pérez-García, Sharma, Bond-Taylor et al., 2024)](https://arxiv.org/abs/2401.10815).
|
17 |
|
18 |
- **Developed by:** Microsoft Health Futures
|
19 |
- **Model type:** Vision transformer
|
@@ -46,7 +46,7 @@ Fine-tuning RAD-DINO is typically not necessary to obtain good performance in do
|
|
46 |
|
47 |
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
48 |
|
49 |
-
##
|
50 |
|
51 |
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
52 |
|
@@ -74,15 +74,17 @@ Underlying biases of the training datasets may not be well characterised.
|
|
74 |
...
|
75 |
>>> # Download the model
|
76 |
>>> repo = "microsoft/rad-dino"
|
77 |
-
>>> model = AutoModel.from_pretrained(repo)
|
78 |
|
79 |
>>> # The processor takes a PIL image, performs resizing, center-cropping, and
|
80 |
>>> # intensity normalization using stats from MIMIC-CXR, and returns a
|
81 |
>>> # dictionary with a PyTorch tensor ready for the encoder
|
82 |
-
>>> processor =
|
83 |
>>>
|
84 |
>>> # Download and preprocess a chest X-ray
|
85 |
>>> image = download_sample_image()
|
|
|
|
|
86 |
>>> inputs = processor(images=image, return_tensors="pt")
|
87 |
>>>
|
88 |
>>> # Encode the image!
|
@@ -90,8 +92,8 @@ Underlying biases of the training datasets may not be well characterised.
|
|
90 |
>>> outputs = model(**inputs)
|
91 |
>>>
|
92 |
>>> # Look at the CLS embeddings
|
93 |
-
>>> cls_embeddings = outputs.pooler_output
|
94 |
-
>>> cls_embeddings # (batch_size, num_channels)
|
95 |
torch.Size([1, 768])
|
96 |
>>>
|
97 |
>>> # Look at the patch embeddings (needs `pip install einops`)
|
@@ -220,4 +222,4 @@ We used [SimpleITK](https://simpleitk.org/) and [Pydicom](https://pydicom.github
|
|
220 |
|
221 |
## Model card contact
|
222 |
|
223 |
-
Fernando Pérez-García ([`[email protected]`](mailto:[email protected])).
|
|
|
13 |
|
14 |
RAD-DINO is a vision transformer model trained to encode chest X-rays using the self-supervised learning method [DINOv2](https://openreview.net/forum?id=a68SUt6zFt).
|
15 |
|
16 |
+
RAD-DINO is described in detail in [RAD-DINO: Exploring Scalable Medical Image Encoders Beyond Text Supervision (F. Pérez-García, H. Sharma, S. Bond-Taylor, et al., 2024)](https://arxiv.org/abs/2401.10815).
|
17 |
|
18 |
- **Developed by:** Microsoft Health Futures
|
19 |
- **Model type:** Vision transformer
|
|
|
46 |
|
47 |
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
48 |
|
49 |
+
## Biases, risks, and limitations
|
50 |
|
51 |
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
52 |
|
|
|
74 |
...
|
75 |
>>> # Download the model
|
76 |
>>> repo = "microsoft/rad-dino"
|
77 |
+
>>> model = AutoModel.from_pretrained(repo)
|
78 |
|
79 |
>>> # The processor takes a PIL image, performs resizing, center-cropping, and
|
80 |
>>> # intensity normalization using stats from MIMIC-CXR, and returns a
|
81 |
>>> # dictionary with a PyTorch tensor ready for the encoder
|
82 |
+
>>> processor = AutoImageProcessor.from_pretrained(repo)
|
83 |
>>>
|
84 |
>>> # Download and preprocess a chest X-ray
|
85 |
>>> image = download_sample_image()
|
86 |
+
>>> image.size # (width, height)
|
87 |
+
(2765, 2505)
|
88 |
>>> inputs = processor(images=image, return_tensors="pt")
|
89 |
>>>
|
90 |
>>> # Encode the image!
|
|
|
92 |
>>> outputs = model(**inputs)
|
93 |
>>>
|
94 |
>>> # Look at the CLS embeddings
|
95 |
+
>>> cls_embeddings = outputs.pooler_output
|
96 |
+
>>> cls_embeddings.shape # (batch_size, num_channels)
|
97 |
torch.Size([1, 768])
|
98 |
>>>
|
99 |
>>> # Look at the patch embeddings (needs `pip install einops`)
|
|
|
222 |
|
223 |
## Model card contact
|
224 |
|
225 |
+
Fernando Pérez-García ([`[email protected]`](mailto:[email protected])).
|