
# LAION Occupation

This dataset is a subset of [LAION-2B-en](https://laion.ai/blog/laion-5b/) containing 1.8M samples, each assigned to one of 153 occupations. This dataset was curated as part of our investigation into gender-occupation biases in LAION presented in [Fair Diffusion](https://arxiv.org/abs/2302.10893).
For downloading the images, check out [img2dataset](https://github.com/rom1504/img2dataset).

## Data Collection

 We identified relevant images in the dataset by computing their CLIP similarity to a textual description of the target occupation. All descriptions were in the form of "an image of the face of a \<<em>occupation</em>\>". Consequently, we included all images above an empirically determined threshold. 

## Probability of faces

The dataset also contains annotations for the probability of a human face being depicted. Scores were calculated using the MTCNN Face Detector of [FaceNet](https://github.com/timesler/facenet-pytorch). Empirically, scores above ca. 0.97 can be reasonably assumed to include recognizable faces.

## Dataset Format

The dataset consists of the following fields:

| Field | Explanation |
| ----------- | ----------- |
| URL | Url of the image.  |
| TEXT | Text caption of the image. |
| occupation   | Identified occupation. |
| pface | Probability of a face being contained in the image as per FaceNet. will be NaN if the image could not be retrieved. |
| url_active  | Whether or not we were able to retrieve the image from the corresponding URL.  |
| retr_sim  | Cosine similarity between CLIP embeddings of image and retrieval prompt. |
| laion_index   | Index of the sample in the original LAION-2B-en. |
| hash  | Usual LAION hash of URL and caption. |
license:cc-by-sa-4.0 arxiv:2302.10893 region:us
{"license": "cc-by-sa-4.0", "pretty_name": "LAION Occupation"}
TAGS
#license-cc-by-sa-4.0 #arxiv-2302.10893 #region-us 

LAION Occupation
================


This dataset is a subset of LAION-2B-en containing 1.8M samples, each assigned to one of 153 occupations. This dataset was curated as part of our investigation into gender-occupation biases in LAION presented in Fair Diffusion.
For downloading the images, check out img2dataset.


Data Collection
---------------


We identified relevant images in the dataset by computing their CLIP similarity to a textual description of the target occupation. All descriptions were in the form of "an image of the face of a <*occupation*>". Consequently, we included all images above an empirically determined threshold.


Probability of faces
--------------------


The dataset also contains annotations for the probability of a human face being depicted. Scores were calculated using the MTCNN Face Detector of FaceNet. Empirically, scores above ca. 0.97 can be reasonably assumed to include recognizable faces.


Dataset Format
--------------


The dataset consists of the following fields:




TAGS
#license-cc-by-sa-4.0 #arxiv-2302.10893 #region-us 

passage: TAGS
#license-cc-by-sa-4.0 #arxiv-2302.10893 #region-us 
