Update README.md

README.md

@@ -19,17 +19,109 @@ model-index:
         value: 0.8050341606140137
 ---
 
-# NASA_GeneLab_MBT
+# NASA GeneLab VisionTransformer on BPS Microscopy Data
 
 NASA GeneLab VisionTransformer on BPS Microscopy Data
 
 ## Example Images
 
-
 #### High_Energy_Ion_Fe_Nuclei
 
 ![High_Energy_Ion_Fe_Nuclei](images/High_Energy_Ion_Fe_Nuclei)
 
 #### XRay_irradiated_Nuclei
 
-![XRay_irradiated_Nuclei](images/XRay_irradiated_Nuclei)
+![XRay_irradiated_Nuclei](images/XRay_irradiated_Nuclei)
+
+## Authors:
+[Frank Soboczenski](https://h21k.github.io/), University of York & King's College London, UK<br>
+[tba]
+[tba]
+[tba]
+
+## General:
+This Vision Transformer model has been fine-tuned on Solar Dynamics Observatory (SDO) data. The images used are available here:
+[Solar Dynamics Observatory Gallery](https://sdo.gsfc.nasa.gov/gallery/main). This is a Vision Transformer model fine-tuned on SDO data in an active region classification task. We aim to highlight the ease of use of the HuggingFace platform, integration with popular deep learning frameworks such as PyTorch, TensorFlow, or JAX, performance monitoring with Weights and Biases, and the ability to effortlessly utilize pre-trained large scale Transformer models for targeted fine-tuning purposes. This is to our knowledge the first Vision Transformer model on NASA SDO mission data and we are working on additional versions to address challenges in this domain. 
+
+<b>The data used was provided courtesy of NASA/SDO and the AIA, EVE, and HMI science teams.
+The authors gratefully acknowledge the entire NASA Solar Dynamics Observatory Mission Team.</b><br>
+
+For the SDO team: this model is the first version for demonstration purposes. It is only trained on the SDO Gallery data atm and we're working on additional data. 
+
+We will include more technical details here soon. 
+
+## Example Images
+--> Use one of the images below for the inference API field on the upper right.
+
+Additional images for testing can be found at: 
+[Solar Dynamics Observatory Gallery](https://sdo.gsfc.nasa.gov/gallery/main)
+
+### NASA_SDO_Coronal_Hole
+
+![NASA_SDO_Coronal_Hole](images/NASA_SDO_Coronal_Hole2.jpg)
+
+### NASA_SDO_Coronal_Loop
+
+![NASA_SDO_Coronal_Loop](images/NASA_SDO_Coronal_Loop.jpg)
+
+## Training data
+The ViT model was pretrained on a dataset consisting of 14 million images and 21k classes ([ImageNet-21k](http://www.image-net.org/).
+More information on the base model used can be found here: (https://huggingface.co/google/vit-base-patch16-224-in21k)
+
+## How to use this Model 
+(quick snippet to work on Google Colab - comment the pip install for local use if you have transformers already installed)
+
+```python
+!pip install transformers --quiet
+from transformers import AutoFeatureExtractor, AutoModelForImageClassification
+from PIL import Image
+import requests
+
+url = 'https://sdo.gsfc.nasa.gov/assets/gallery/preview/211_coronalhole.jpg'
+image = Image.open(requests.get(url, stream=True).raw)
+
+feature_extractor = AutoFeatureExtractor.from_pretrained("kenobi/SDO_VT1")
+model = AutoModelForImageClassification.from_pretrained("kenobi/SDO_VT1")
+inputs = feature_extractor(images=image, return_tensors="pt")
+
+outputs = model(**inputs)
+logits = outputs.logits
+# model predicts one of the three fine-tuned classes (NASA_SDO_Coronal_Hole, NASA_SDO_Coronal_Loop or NASA_SDO_Solar_Flare) 
+predicted_class_idx = logits.argmax(-1).item()
+print("Predicted class:", model.config.id2label[predicted_class_idx])
+```
+
+## BibTeX & References
+
+A publication on this work is currently in preparation. In the meantime, please refer to this model by using the following citation: 
+
+For the base ViT model used please refer to:
+
+```bibtex
+@misc{wu2020visual,
+      title={Visual Transformers: Token-based Image Representation and Processing for Computer Vision}, 
+      author={Bichen Wu and Chenfeng Xu and Xiaoliang Dai and Alvin Wan and Peizhao Zhang and Zhicheng Yan and Masayoshi Tomizuka and Joseph Gonzalez and Kurt Keutzer and Peter Vajda},
+      year={2020},
+      eprint={2006.03677},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```
+For referring to Imagenet:
+
+```bibtex
+@inproceedings{deng2009imagenet,
+  title={Imagenet: A large-scale hierarchical image database},
+  author={Deng, Jia and Dong, Wei and Socher, Richard and Li, Li-Jia and Li, Kai and Fei-Fei, Li},
+  booktitle={2009 IEEE conference on computer vision and pattern recognition},
+  pages={248--255},
+  year={2009},
+  organization={Ieee}
+}
+```
+
+
+
+
+
+