Update README.md

README.md

@@ -15,82 +15,71 @@ tags:
 ---
 
 # Model Card for X3D-KABR-Kinetics
-X3D-KABR-Kinetics is a behavior recognition 
+X3D-KABR-Kinetics is a behavior recognition model for in situ drone videos of zebras and giraffes, 
+built using X3D model initialized on Kinetics weights. 
+It is trained on the [KABR dataset](https://huggingface.co/datasets/imageomics/KABR), 
+which is comprised of 10 hours of aerial video footage of reticulated giraffes
+(*Giraffa reticulata*), Plains zebras (*Equus quagga*), and Grevy’s zebras
+(*Equus grevyi*) captured using a DJI Mavic 2S drone.
+It includes both spatiotemporal (i.e., mini-scenes) and behavior annotations provided by an expert
+behavioral ecologist.
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
+- **Developed by:** [Maksim Kholiavchenko, Maksim Kukushkin, Otto Brookes, Jenna Kline, Sam Stevens, Isla Duporge, Alec Sheets,
+Reshma R. Babu, Namrata Banerji, Elizabeth Campolongo,
+Matthew Thompson, Nina Van Tiel, Jackson Miliko,
+Eduardo Bessa Mirmehdi, Thomas Schmid,
+Tanya Berger-Wolf, Daniel I. Rubenstein, Tilo Burghardt, Charles V. Stewart]
 
-- **Developed by:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed -- choose a license (see above notes)]
-- **Fine-tuned from model:** [More Information Needed]
+- **Model type:** [X3D]
+- **License:** [MIT]
+- **Fine-tuned from model:** [X3D-S, Kinetics]
 
-### Model Sources
+This model was developed for the benefit of the community as an open-source product, thus we request that any derivative products are also open-source.
 
-<!-- Provide the basic links for the model. -->
+### Model Sources
 
-- **Repository:** [Project Repo]
-- **Paper:** [More Information Needed--optional]
-- **Demo:** [More Information Needed--encouraged]
+- **Repository:** [Project Repo](https://github.com/Imageomics/kabr-tools)
+- **Paper:** [Paper Link](https://openaccess.thecvf.com/content/WACV2024W/CV4Smalls/papers/Kholiavchenko_KABR_In-Situ_Dataset_for_Kenyan_Animal_Behavior_Recognition_From_Drone_WACVW_2024_paper.pdf)
+- **Demo** [Project Page](https://kabrdata.xyz/)
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+X3D-KABR-Kinetics has extensively studied ungulate behavior classification from aerial video.
 
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
+Please look at the [demo](https://github.com/Imageomics/kabr-tools) here for examples of how this model can generate time-budgets from aerial video of animals.
 
-### Downstream Use
-
-<!-- [optional] This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+This model was trained to detect and classify behavior from drone videos of zebras and giraffes in Kenya. It may not perform well on other species or settings.
 
+<!-- 
 ## Bias, Risks, and Limitations
 
 <!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
+<!-- 
 [More Information Needed]
 
 ### Recommendations
 
 <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+This model was trained to detect and classify behavior from drone videos of zebras and giraffes in Kenya. It may not perform well on other species or settings.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-<!-- Put code here or links to files to run. Set up code blocks like this:
-```
-<code here>
-```
--->
-
-[More Information Needed]
+Please take a look at the [demo](https://github.com/Imageomics/kabr-tools) to get started with the model.
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card where possible, otherwise link to the original source with more info. 
-Provide a basic overview of the training data and documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+[KABR Dataset](https://huggingface.co/datasets/imageomics/KABR)
 
 ### Training Procedure 
 
@@ -98,23 +87,37 @@ Provide a basic overview of the training data and documentation related to data
 
 #### Preprocessing
 
-[More Information Needed--encouraged]
+Raw drone videos were pre-processed using CVAT to detect and track each individual animal in each
+high-resolution video and link the results into tracklets. 
+For each tracklet, we create a separate video, called a mini-scene, by extracting a sub-image centered on each
+detection in a video frame. 
+This allows us to compensate for the drone's movement and provides a stable, zoomed-in representation of the animal.
+
+See [project page](https://kabrdata.xyz/) and the [paper](https://openaccess.thecvf.com/content/WACV2024W/CV4Smalls/papers/Kholiavchenko_KABR_In-Situ_Dataset_for_Kenyan_Animal_Behavior_Recognition_From_Drone_WACVW_2024_paper.pdf) for data preprocessing details.
 
+We applied data augmentation techniques during training, including horizontal flipping to randomly
+mirror the input frames horizontally and color augmentations to randomly modify the
+brightness, contrast, and saturation of the input frames.
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+The model was trained for 120 epochs, using a batch size of 5.
+We used the EQL loss function to address the long-tailed class distribution and SGD optimizer with a learning rate of 1e5.
+We used a sample rate of 16x5, and random weight initialization.
 
+<!-- ADD RESULTS ONCE NEW PAPER PUBLISHED
+
+<!-- 
 #### Speeds, Sizes, Times 
 
 <!-- [optional] This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
+<!-- 
 [More Information Needed]
 
 ## Evaluation
 
 <!-- This section describes the evaluation protocols and provides the results. -->
-
+<!-- 
 [More Information Needed]
 
 ### Testing Data, Factors & Metrics
@@ -123,19 +126,19 @@ Provide a basic overview of the training data and documentation related to data
 
 <!-- This should link to a Dataset Card if possible, otherwise link to the original source with more info.
 Provide a basic overview of the test data and documentation related to any data pre-processing or additional filtering. -->
-
+<!-- 
 [More Information Needed]
 
 #### Factors
 
 <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
+<!-- 
 [More Information Needed]
 
 #### Metrics
 
 <!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
+<!-- 
 [More Information Needed]
 
 ### Results
@@ -149,7 +152,7 @@ Provide a basic overview of the test data and documentation related to any data
 ## Model Examination
 
 <!-- [optional] Relevant interpretability work for the model goes here -->
-
+<!-- 
 [More Information Needed]
 
 ## Environmental Impact
@@ -158,8 +161,9 @@ Provide a basic overview of the test data and documentation related to any data
 It would be great to try to include this.
 
 Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://doi.org/10.48550/arXiv.1910.09700).
+<!-- 
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) 
+presented in [Lacoste et al. (2019)](https://doi.org/10.48550/arXiv.1910.09700).
 
 - **Hardware Type:** [More Information Needed]
 - **Hours used:** [More Information Needed]
@@ -191,62 +195,83 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 <!-- If there is a paper introducing the model, the Bibtex information for that should go in this section. 
 
 See notes at top of file about selecting a license. 
-If you choose CC0: This model is dedicated to the public domain for the benefit of scientific pursuits. We ask that you cite the model and journal paper using the below citations if you make use of it in your research.
+If you choose CC0: This model is dedicated to the public domain for the benefit of scientific pursuits. 
+We ask that you cite the model and journal paper using the below citations if you make use of it in your research.
 
 -->
 
 **BibTeX:**
 
-[More Information Needed]
-<!--
-Replace "<>"s with your info:
 
 If you use our model in your work, please cite the model and associated paper.
 
 **Model**
 ```
-@software{<ref_code>,
-  author = {<author1 and author2>},
+@software{kabr_x3d_model,
+  author = {Maksim Kholiavchenko, Maksim Kukushkin, Otto Brookes, Jenna Kline, Sam Stevens, Isla Duporge, Alec Sheets,
+Reshma R. Babu, Namrata Banerji, Elizabeth Campolongo,
+Matthew Thompson, Nina Van Tiel, Jackson Miliko,
+Eduardo Bessa Mirmehdi, Thomas Schmid,
+Tanya Berger-Wolf, Daniel I. Rubenstein, Tilo Burghardt, Charles V. Stewart},
   doi = {<doi once generated>},
-  title = {<title>},
-  version = {<version#>},
-  year = {<year>},
-  url = {https://huggingface.co/imageomics/<model_name>}
+  title = {KABR model},
+  version = {v0.1},
+  year = {2024},
+  url = {https://huggingface.co/imageomics/x3d-kabr-kinetics}
 }
 ```
 
--for an associated paper:
 **Paper**
 ```
-@article{<ref_code>,
-  title    = {<title>},
-  author   = {<author1 and author2>},
-  journal  = {<journal_name>},
-  year     =  <year>,
-  url      = {<DOI_URL>},
-  doi      = {<DOI>}
+@InProceedings{Kholiavchenko_2024_WACV,
+    author    = {Kholiavchenko, Maksim and Kline, Jenna and Ramirez, Michelle and Stevens, Sam and Sheets, Alec and Babu, Reshma and Banerji, Namrata and Campolongo, Elizabeth and Thompson, Matthew and Van Tiel, Nina and Miliko, Jackson and Bessa, Eduardo and Duporge, Isla and Berger-Wolf, Tanya and Rubenstein, Daniel and Stewart, Charles},
+    title     = {KABR: In-Situ Dataset for Kenyan Animal Behavior Recognition From Drone Videos},
+    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops},
+    month     = {January},
+    year      = {2024},
+    pages     = {31-40}
 }
 ```
--->
+<!-- ADD ONCE PAPER PUBLISHED
+@article{kabr2024,
+  title    = {Deep Dive into KABR: A Dataset for Understanding Ungulate Behavior from In-Situ Drone Video},
+  author   = {Maksim Kholiavchenko, Jenna Kline, Maksim Kukushkin, Otto Brookes, Sam Stevens, Isla Duporge, Alec Sheets,
+Reshma R. Babu, Namrata Banerji, Elizabeth Campolongo,
+Matthew Thompson, Nina Van Tiel, Jackson Miliko,
+Eduardo Bessa Mirmehdi, Thomas Schmid,
+Tanya Berger-Wolf, Daniel I. Rubenstein, Tilo Burghardt, Charles V. Stewart},
+  journal  = {},
+  year     =  2024,
+  url      = {},
+  doi      = {<DOI>}
+}
+
 
 
 ## Acknowledgements
 
-This work was supported by the [Imageomics Institute](https://imageomics.org), which is funded by the US National Science Foundation's Harnessing the Data Revolution (HDR) program under [Award #2118240](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118240) (Imageomics: A New Frontier of Biological Information Powered by Knowledge-Guided Machine Learning). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
+This work was supported by the [Imageomics Institute](https://imageomics.org), 
+which is funded by the US National Science Foundation's Harnessing the Data Revolution (HDR) program under 
+[Award #2118240](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118240) 
+(Imageomics: A New Frontier of Biological Information Powered by Knowledge-Guided Machine Learning). 
+Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) 
+and do not necessarily reflect the views of the National Science Foundation.
 
+<!-- 
 ## Glossary 
 
 <!-- [optional] If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
+<!-- 
 ## More Information 
 
 <!-- [optional] Any other relevant information that doesn't fit elsewhere. -->
 
+
 ## Model Card Authors
 
-[More Information Needed]
+[Jenna Kline and Maksim Kholiavchenko]
 
 ## Model Card Contact
 
-[More Information Needed--optional]
+Maksim Kholiavchenko 
 <!-- Could include who to contact with questions, but this is also what the "Discussions" tab is for. -->