
## Dataset Description

- **Homepage:** [DiffusionFER homepage](https://kdhht2334.github.io/)
- **Repository:** [DiffusionFER repository](https://github.com/kdhht2334/Facial-Expression-Recognition-Zoo)
- **Distribution:** [DiffusionFER Hugging Face Dataset](https://huggingface.co/datasets/FER-Universe/DiffusionFER)
- **Point of Contact:** [Daeha Kim](mailto:kdhht5022@gmail.com)


### Summary

DiffusionFER is the large-scale text-to-image prompt database for face-related tasks. It contains about **1M(ongoing)** images generated by [Stable Diffusion](https://github.com/camenduru/stable-diffusion-webui-colab) using prompt(s) and other parameters.

DiffusionFER is available at [🤗 Hugging Face Dataset](https://huggingface.co/datasets/FER-Universe/DiffusionFER).


### Downstream Tasks and Leaderboards

This DiffusionFER dataset can be utilized for the following downstream tasks.
  - Face detection
  - Facial expression recognition
  - Text-to-emotion prompting

In addition, the virtual subjects included in this dataset provide opportunities to perform various vision tasks related to face privacy.


### Data Loading

DiffusionFER can be loaded via both Python and Git. Please refer Hugging Face [`Datasets`](https://huggingface.co/docs/datasets/quickstart).

```python
from datasets import load_dataset

dataset = load_dataset("FER-Universe/DiffusionFER")
```

```bash
git lfs install
git clone https://huggingface.co/datasets/FER-Universe/DiffusionFER
```

### Pre-trained model

You can easily download and use pre-trained __Swin Transformer__ model with the `Diffusion_Emotion_S` dataset.

Later, Transformer models with the `Diffusion_Emotion_M` or `Diffusion_Emotion_L` will be released.

```python
from transformers import AutoFeatureExtractor, AutoModelForImageClassification

extractor = AutoFeatureExtractor.from_pretrained("kdhht2334/autotrain-diffusion-emotion-facial-expression-recognition-40429105176")

model = AutoModelForImageClassification.from_pretrained("kdhht2334/autotrain-diffusion-emotion-facial-expression-recognition-40429105176")
```

Or just clone the model repo


```bash
git lfs install
git clone https://huggingface.co/kdhht2334/autotrain-diffusion-emotion-facial-expression-recognition-40429105176
```

  - Quick links: [huggingface model documentation](https://huggingface.co/docs/transformers/main/en/model_doc/swin#transformers.SwinForImageClassification)



### Sample Gallery

▼Happy

![Gallery(happy)](https://drive.google.com/uc?id=10YW9XHXFJ9cjutis9Pwpgd0ld6JI84P3)


▼Angry

![Gallery(happy)](https://drive.google.com/uc?id=14qbmOgzqqXGxkatjMfqaUmf0xYwDz--g)


### Subsets

DiffusionFER supports a total of three distinct splits. And, each split additionally provides a face region cropped by [face detector](https://github.com/timesler/facenet-pytorch).
   - DifussionEmotion_S (small), DifussionEmotion_M (medium), DifussionEmotion_L (large).

|Subset|Num of Images|Size|Image Directory |
|:--|--:|--:|--:|
|DifussionEmotion_S (original) | 1.5K | 647M | `DifussionEmotion_S/` |
|DifussionEmotion_S (cropped)  | 1.5K | 322M | `DiffusionEmotion_S_cropped/` |
|DifussionEmotion_M (original) | N/A  | N/A  | `DifussionEmotion_M/` |
|DifussionEmotion_M (cropped)  | N/A  | N/A  | `DiffusionEmotion_M_cropped/` |
|DifussionEmotion_L (original) | N/A  | N/A  | `DifussionEmotion_L/` |
|DifussionEmotion_L (cropped)  | N/A  | N/A  | `DiffusionEmotion_L_cropped/` |


## Dataset Structure

We provide DiffusionFER using a modular file structure. `DiffusionEmotion_S`, the smallest scale, contains about 1,500 images and is divided into folders of a total of 7 emotion classes. The class labels of all these images are included in `dataset_sheet.csv`.
   - In `dataset_sheet.csv`, not only 7-emotion class but also _valence-arousal_ value are annotated.

```bash
# Small version of DB
./
├── DifussionEmotion_S
│   ├── angry
│   │   ├── aaaaaaaa_6.png
│   │   ├── andtcvhp_6.png
│   │   ├── azikakjh_6.png
│   │   ├── [...]
│   ├── fear
│   ├── happy
│   ├── [...]
│   └── surprise
└── dataset_sheet.csv
```

- Middle size DB will be uploaded soon.
```bash
# Medium version of DB
(ongoing)
```

- TBD
```bash
# Large version of DB
(ongoing)
```


### Prompt Format

Basic format is as follows: "`Emotion`, `Race` `Age` style, a realistic portrait of `Style` `Gender`, upper body, `Others`".

- ex) one person, neutral emotion, white middle-aged style, a realistic portrait of man, upper body

Examples of format categories are listed in the table below.

| Category | Prompt(s) |
| --- | --- |
| `Emotion` | neutral emotion<br>happy emotion, with open mouth, smiley<br>sad emotion, with tears, lowered head, droopy eyebrows<br>surprise emotion, with open mouth, big eyes<br>fear emotion, scared, haunted<br>disgust emotion, frown, angry expression with open mouth<br>angry emotion, with open mouth, frown eyebrow, fierce, furious | 
| `Race`    | white<br>black<br>latin | 
| `Age`     | teen<br>middle-aged<br>old |
| `Gender`  | man<br>woman |
| `Style`   | gentle<br>handsome<br>pretty<br>cute<br>mature<br>punky<br>freckles<br>beautiful crystal eyes<br>big eyes<br>small nose<br>... |
| `Others`  | 4K<br>8K<br>cyberpunk<br>camping<br>ancient<br>medieval Europe<br>... |


### Prompt Engineering

You can improve the performance and quality of generating default prompts with the settings below.
   
```
{
  "negative prompt": "sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, backlight, (duplicate:1.331), (morbid:1.21), (mutilated:1.21), mutated hands, (poorly drawn hands:1.331), (bad anatomy:1.21), (bad proportions:1.331), extra limbs, (disfigured:1.331), (missing arms:1.331), (extra legs:1.331), (fused fingers:1.61051), (too many fingers:1.61051), (unclear eyes:1.331), bad hands, missing fingers, extra digit",
  "steps": 50,
  "sampling method": "DPM++ 2M Karras"
  "Width": "512",
  "Height": "512",
  "CFG scale": 12.0,
  "seed": -1,
}
```

### Annotations

The DiffusionFER contains annotation process both 7-emotion classes and valence-arousal values.

#### Annotation process

This process was carried out inspired by the theory of the two research papers below.

  - JA Russell, [A circumplex model of affect](https://d1wqtxts1xzle7.cloudfront.net/38425675/Russell1980-libre.pdf?1439132613=&response-content-disposition=inline%3B+filename%3DRussell1980.pdf&Expires=1678595455&Signature=UtbPsezND6w8vbISBiuL-ECk6hDI0etLcJSE7kJMC~hAkMSu9YyQcPKdVpdHSSq7idfcQ~eEKsqptvYpy0199DX0gi-nHJwhsciahC-zgDwylEUo6ykhP6Ab8VWCOW-DM21jHNvbYLQf7Pwi66fGvm~5bAXPc1o4HHpQpk-Cr7b0tW9lYnl3qgLoVeIICg6FLu0elbtVztgH5OS1uL6V~QhiP2PCwZf~WCHuJRQrWdPt5Kuco0lsNr1Qikk1~d7HY3ZcUTRZcMNDdem8XAFDH~ak3QER6Ml~JDkNFcLuygz~tjL4CdScVhByeAuMe3juyijtBFtYWH2h30iRkUDalg__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA)
  - A Mollahosseini et al., [AffectNet](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8013713&casa_token=C3QmhmiB6Y8AAAAA:1CiUll0bhIq06M17YwFIvxuse7GOosEN9G1A8vxVzR8Vb5eaFp6ERIjg7xhSIQlf008KLsfJ-w&tag=1)

#### Who are the annotators?

[Daeha Kim](mailto:kdhht5022@gmail.com) and [Dohee Kang](mailto:asrs777@naver.com)


## Additional Information

### Dataset Curators

DiffusionFER is created by [Daeha Kim](https://kdhht2334.github.io/) and [Dohee Kang](https://github.com/KangDohee2270).

### Acknowledgments

This repository is heavily inspired by [DiffusionDB](https://huggingface.co/datasets/poloclub/diffusiondb), with some format references. Thank you for your interest in [DiffusionDB](https://huggingface.co/datasets/poloclub/diffusiondb).

### Licensing Information

The DiffusionFER is available under the [CC0 1.0 License](https://creativecommons.org/publicdomain/zero/1.0/). NOTE: The primary purpose of this dataset is research. We are not responsible if you take any other action using this dataset.

### Contributions

If you have any questions, feel free to [open an issue](https://github.com/kdhht2334/Facial-Expression-Recognition-Zoo/issues/new) or contact [Daeha Kim](https://kdhht2334.github.io/).
task_categories:text-to-image task_ids:image-captioning task_ids:face-detection annotations_creators:no-annotation language_creators:found size_categories:n<500MB source_datasets:original language:en license:cc0-1.0 stable diffusion prompt engineering prompts research paper facial expression recognition emotion recognition region:us
{"annotations_creators": ["no-annotation"], "language_creators": ["found"], "language": ["en"], "license": "cc0-1.0", "size_categories": ["n<500MB"], "source_datasets": ["original"], "task_categories": ["text-to-image"], "task_ids": ["image-captioning", "face-detection"], "pretty_name": "DiffusionFER", "layout": "default", "title": "Home", "nav_order": 1, "has_children": false, "tags": ["stable diffusion", "prompt engineering", "prompts", "research paper", "facial expression recognition", "emotion recognition"]}
TAGS
#task_categories-text-to-image #task_ids-image-captioning #task_ids-face-detection #annotations_creators-no-annotation #language_creators-found #size_categories-n<500MB #source_datasets-original #language-English #license-cc0-1.0 #stable diffusion #prompt engineering #prompts #research paper #facial expression recognition #emotion recognition #region-us 

Dataset Description
-------------------


* Homepage: DiffusionFER homepage
* Repository: DiffusionFER repository
* Distribution: DiffusionFER Hugging Face Dataset
* Point of Contact: Daeha Kim


### Summary


DiffusionFER is the large-scale text-to-image prompt database for face-related tasks. It contains about 1M(ongoing) images generated by Stable Diffusion using prompt(s) and other parameters.


DiffusionFER is available at Hugging Face Dataset.


### Downstream Tasks and Leaderboards


This DiffusionFER dataset can be utilized for the following downstream tasks.


* Face detection
* Facial expression recognition
* Text-to-emotion prompting


In addition, the virtual subjects included in this dataset provide opportunities to perform various vision tasks related to face privacy.


### Data Loading


DiffusionFER can be loaded via both Python and Git. Please refer Hugging Face 'Datasets'.


### Pre-trained model


You can easily download and use pre-trained **Swin Transformer** model with the 'Diffusion\_Emotion\_S' dataset.


Later, Transformer models with the 'Diffusion\_Emotion\_M' or 'Diffusion\_Emotion\_L' will be released.


Or just clone the model repo


* Quick links: huggingface model documentation


### Sample Gallery


▼Happy


!Gallery(happy)


▼Angry


!Gallery(happy)


### Subsets


DiffusionFER supports a total of three distinct splits. And, each split additionally provides a face region cropped by face detector.


* DifussionEmotion\_S (small), DifussionEmotion\_M (medium), DifussionEmotion\_L (large).



Dataset Structure
-----------------


We provide DiffusionFER using a modular file structure. 'DiffusionEmotion\_S', the smallest scale, contains about 1,500 images and is divided into folders of a total of 7 emotion classes. The class labels of all these images are included in 'dataset\_sheet.csv'.


* In 'dataset\_sheet.csv', not only 7-emotion class but also *valence-arousal* value are annotated.
* Middle size DB will be uploaded soon.
* TBD


### Prompt Format


Basic format is as follows: "'Emotion', 'Race' 'Age' style, a realistic portrait of 'Style' 'Gender', upper body, 'Others'".


* ex) one person, neutral emotion, white middle-aged style, a realistic portrait of man, upper body


Examples of format categories are listed in the table below.



### Prompt Engineering


You can improve the performance and quality of generating default prompts with the settings below.


### Annotations


The DiffusionFER contains annotation process both 7-emotion classes and valence-arousal values.


#### Annotation process


This process was carried out inspired by the theory of the two research papers below.


* JA Russell, A circumplex model of affect
* A Mollahosseini et al., AffectNet


#### Who are the annotators?


Daeha Kim and Dohee Kang


Additional Information
----------------------


### Dataset Curators


DiffusionFER is created by Daeha Kim and Dohee Kang.


### Acknowledgments


This repository is heavily inspired by DiffusionDB, with some format references. Thank you for your interest in DiffusionDB.


### Licensing Information


The DiffusionFER is available under the CC0 1.0 License. NOTE: The primary purpose of this dataset is research. We are not responsible if you take any other action using this dataset.


### Contributions


If you have any questions, feel free to open an issue or contact Daeha Kim.



### Summary


DiffusionFER is the large-scale text-to-image prompt database for face-related tasks. It contains about 1M(ongoing) images generated by Stable Diffusion using prompt(s) and other parameters.


DiffusionFER is available at Hugging Face Dataset. ### Downstream Tasks and Leaderboards


This DiffusionFER dataset can be utilized for the following downstream tasks.


* Face detection
* Facial expression recognition
* Text-to-emotion prompting


In addition, the virtual subjects included in this dataset provide opportunities to perform various vision tasks related to face privacy. ### Data Loading


DiffusionFER can be loaded via both Python and Git. Please refer Hugging Face 'Datasets'. ### Pre-trained model


You can easily download and use pre-trained **Swin Transformer** model with the 'Diffusion\_Emotion\_S' dataset.


Later, Transformer models with the 'Diffusion\_Emotion\_M' or 'Diffusion\_Emotion\_L' will be released.


Or just clone the model repo


* Quick links: huggingface model documentation ### Sample Gallery


▼Happy


!Gallery(happy)


▼Angry


!Gallery(happy) ### Subsets


DiffusionFER supports a total of three distinct splits. And, each split additionally provides a face region cropped by face detector.


* DifussionEmotion\_S (small), DifussionEmotion\_M (medium), DifussionEmotion\_L (large).



Dataset Structure
-----------------


We provide DiffusionFER using a modular file structure. 'DiffusionEmotion\_S', the smallest scale, contains about 1,500 images and is divided into folders of a total of 7 emotion classes. The class labels of all these images are included in 'dataset\_sheet.csv'.


* In 'dataset\_sheet.csv', not only 7-emotion class but also *valence-arousal* value are annotated.
* Middle size DB will be uploaded soon.
* TBD ### Prompt Format


Basic format is as follows: "'Emotion', 'Race' 'Age' style, a realistic portrait of 'Style' 'Gender', upper body, 'Others'".


* ex) one person, neutral emotion, white middle-aged style, a realistic portrait of man, upper body


Examples of format categories are listed in the table below. ### Prompt Engineering


You can improve the performance and quality of generating default prompts with the settings below. ### Annotations


The DiffusionFER contains annotation process both 7-emotion classes and valence-arousal values. #### Annotation process


This process was carried out inspired by the theory of the two research papers below.


* JA Russell, A circumplex model of affect
* A Mollahosseini et al., AffectNet #### Who are the annotators?


Daeha Kim and Dohee Kang


Additional Information
---------------------- ### Dataset Curators


DiffusionFER is created by Daeha Kim and Dohee Kang. ### Acknowledgments


This repository is heavily inspired by DiffusionDB, with some format references. Thank you for your interest in DiffusionDB. ### Licensing Information


The DiffusionFER is available under the CC0 1.0 License. NOTE: The primary purpose of this dataset is research. We are not responsible if you take any other action using this dataset. ### Contributions


If you have any questions, feel free to open an issue or contact Daeha Kim.
TAGS
#task_categories-text-to-image #task_ids-image-captioning #task_ids-face-detection #annotations_creators-no-annotation #language_creators-found #size_categories-n<500MB #source_datasets-original #language-English #license-cc0-1.0 #stable diffusion #prompt engineering #prompts #research paper #facial expression recognition #emotion recognition #region-us 
 ### Summary


DiffusionFER is the large-scale text-to-image prompt database for face-related tasks. It contains about 1M(ongoing) images generated by Stable Diffusion using prompt(s) and other parameters.


DiffusionFER is available at Hugging Face Dataset. ### Downstream Tasks and Leaderboards


This DiffusionFER dataset can be utilized for the following downstream tasks.


* Face detection
* Facial expression recognition
* Text-to-emotion prompting


In addition, the virtual subjects included in this dataset provide opportunities to perform various vision tasks related to face privacy. ### Data Loading


DiffusionFER can be loaded via both Python and Git. Please refer Hugging Face 'Datasets'. ### Pre-trained model


You can easily download and use pre-trained **Swin Transformer** model with the 'Diffusion\_Emotion\_S' dataset.


Later, Transformer models with the 'Diffusion\_Emotion\_M' or 'Diffusion\_Emotion\_L' will be released.


Or just clone the model repo


* Quick links: huggingface model documentation ### Sample Gallery


▼Happy


!Gallery(happy)


▼Angry


!Gallery(happy) ### Subsets


DiffusionFER supports a total of three distinct splits. And, each split additionally provides a face region cropped by face detector.


* DifussionEmotion\_S (small), DifussionEmotion\_M (medium), DifussionEmotion\_L (large).



Dataset Structure
-----------------


We provide DiffusionFER using a modular file structure. 'DiffusionEmotion\_S', the smallest scale, contains about 1,500 images and is divided into folders of a total of 7 emotion classes. The class labels of all these images are included in 'dataset\_sheet.csv'.


* In 'dataset\_sheet.csv', not only 7-emotion class but also *valence-arousal* value are annotated.
* Middle size DB will be uploaded soon.
* TBD ### Prompt Format


Basic format is as follows: "'Emotion', 'Race' 'Age' style, a realistic portrait of 'Style' 'Gender', upper body, 'Others'".


* ex) one person, neutral emotion, white middle-aged style, a realistic portrait of man, upper body


Examples of format categories are listed in the table below. ### Prompt Engineering


You can improve the performance and quality of generating default prompts with the settings below. ### Annotations


The DiffusionFER contains annotation process both 7-emotion classes and valence-arousal values. #### Annotation process


This process was carried out inspired by the theory of the two research papers below.


* JA Russell, A circumplex model of affect
* A Mollahosseini et al., AffectNet #### Who are the annotators?


Daeha Kim and Dohee Kang


Additional Information
---------------------- ### Dataset Curators


DiffusionFER is created by Daeha Kim and Dohee Kang. ### Acknowledgments


This repository is heavily inspired by DiffusionDB, with some format references. Thank you for your interest in DiffusionDB. ### Licensing Information


The DiffusionFER is available under the CC0 1.0 License. NOTE: The primary purpose of this dataset is research. We are not responsible if you take any other action using this dataset. ### Contributions


If you have any questions, feel free to open an issue or contact Daeha Kim.
passage: TAGS
#task_categories-text-to-image #task_ids-image-captioning #task_ids-face-detection #annotations_creators-no-annotation #language_creators-found #size_categories-n<500MB #source_datasets-original #language-English #license-cc0-1.0 #stable diffusion #prompt engineering #prompts #research paper #facial expression recognition #emotion recognition #region-us 
### Summary


DiffusionFER is the large-scale text-to-image prompt database for face-related tasks. It contains about 1M(ongoing) images generated by Stable Diffusion using prompt(s) and other parameters.


DiffusionFER is available at Hugging Face Dataset.### Downstream Tasks and Leaderboards


This DiffusionFER dataset can be utilized for the following downstream tasks.


* Face detection
* Facial expression recognition
* Text-to-emotion prompting


In addition, the virtual subjects included in this dataset provide opportunities to perform various vision tasks related to face privacy.### Data Loading


DiffusionFER can be loaded via both Python and Git. Please refer Hugging Face 'Datasets'.### Pre-trained model


You can easily download and use pre-trained **Swin Transformer** model with the 'Diffusion\_Emotion\_S' dataset.


Later, Transformer models with the 'Diffusion\_Emotion\_M' or 'Diffusion\_Emotion\_L' will be released.


Or just clone the model repo


* Quick links: huggingface model documentation### Sample Gallery


▼Happy


!Gallery(happy)


▼Angry


!Gallery(happy)