
# Dataset Card for Pick-a-Pic (v1)

## Dataset Description

- **Homepage: The web app can be found at [pickapic.io](https://pickapic.io/)**
- **Repository: The repository of [PickScore](https://github.com/yuvalkirstain/PickScore)**
- **Paper: [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://arxiv.org/abs/2305.01569).**
- **Leaderboard: TODO **
- **Point of Contact: TODO **

### Dataset Summary

The Pick-a-Pic dataset was collected with the [Pick-a-Pic web app](https://pickapic.io/) and contains over half-a-million examples of human preferences over model-generated images.
This dataset with URLs instead of the actual images (which makes it much smaller in size) can be found [here](https://huggingface.co/datasets/yuvalkirstain/pickapic_v1_no_images).

See the corresponding paper [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://arxiv.org/abs/2305.01569) for more details.

If you want to download this dataset with URLs instead of images to save space, please see [this version of the dataset](https://huggingface.co/datasets/yuvalkirstain/pickapic_v1_no_images).

### Supported Tasks and Leaderboards

Task: Select preferred image in test-set.
| **Models** | **Test-Set Accuracy (%)** |
| --- | --- |
| [PickScore](https://arxiv.org/abs/2305.01569) | 70.2% |
| Human Expert Baseline | 68.0% |
| [HPS](https://arxiv.org/abs/2303.14420) | 66.7% |
| [ImageReward](https://arxiv.org/abs/2304.05977) | 61.1% |
| [CLIP-H](https://arxiv.org/abs/2210.03927) | 60.8% |
| [Aesthetics](https://arxiv.org/abs/2210.08402) | 56.8% |

### Data Splits

The dataset has three main splits: train, validation, validation_unique (with one example per prompt), test, and test_unique.


### Citation Information
If you find this work useful, please cite:
```bibtex
@inproceedings{Kirstain2023PickaPicAO,
  title={Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation},
  author={Yuval Kirstain and Adam Polyak and Uriel Singer and Shahbuland Matiana and Joe Penna and Omer Levy},
  year={2023}
}
```

### LICENSE
MIT License

Copyright (c) 2021

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

arxiv:2305.01569 arxiv:2303.14420 arxiv:2304.05977 arxiv:2210.03927 arxiv:2210.08402 region:us
{"dataset_info": {"features": [{"name": "are_different", "dtype": "bool"}, {"name": "best_image_uid", "dtype": "string"}, {"name": "caption", "dtype": "string"}, {"name": "created_at", "dtype": "timestamp[ns]"}, {"name": "has_label", "dtype": "bool"}, {"name": "image_0_uid", "dtype": "string"}, {"name": "image_0_url", "dtype": "string"}, {"name": "image_1_uid", "dtype": "string"}, {"name": "image_1_url", "dtype": "string"}, {"name": "jpg_0", "dtype": "binary"}, {"name": "jpg_1", "dtype": "binary"}, {"name": "label_0", "dtype": "float64"}, {"name": "label_1", "dtype": "float64"}, {"name": "model_0", "dtype": "string"}, {"name": "model_1", "dtype": "string"}, {"name": "ranking_id", "dtype": "int64"}, {"name": "user_id", "dtype": "int64"}, {"name": "num_example_per_prompt", "dtype": "int64"}, {"name": "__index_level_0__", "dtype": "int64"}], "splits": [{"name": "train", "num_bytes": 193273338802, "num_examples": 583747}, {"name": "validation", "num_bytes": 5638295249, "num_examples": 17439}, {"name": "test", "num_bytes": 4621428929, "num_examples": 14073}, {"name": "validation_unique", "num_bytes": 178723392, "num_examples": 500}, {"name": "test_unique", "num_bytes": 178099641, "num_examples": 500}], "download_size": 202289408791, "dataset_size": 203889886013}}
TAGS
#arxiv-2305.01569 #arxiv-2303.14420 #arxiv-2304.05977 #arxiv-2210.03927 #arxiv-2210.08402 #region-us 

Dataset Card for Pick-a-Pic (v1)
================================


Dataset Description
-------------------


* Homepage: The web app can be found at URL
* Repository: The repository of PickScore
* Paper: Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation.
* Leaderboard: TODO
* Point of Contact: TODO


### Dataset Summary


The Pick-a-Pic dataset was collected with the Pick-a-Pic web app and contains over half-a-million examples of human preferences over model-generated images.
This dataset with URLs instead of the actual images (which makes it much smaller in size) can be found here.


See the corresponding paper Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation for more details.


If you want to download this dataset with URLs instead of images to save space, please see this version of the dataset.


### Supported Tasks and Leaderboards


Task: Select preferred image in test-set.



### Data Splits


The dataset has three main splits: train, validation, validation\_unique (with one example per prompt), test, and test\_unique.


If you find this work useful, please cite:


### LICENSE


MIT License


Copyright (c) 2021


Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:


The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.


THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



### Dataset Summary


The Pick-a-Pic dataset was collected with the Pick-a-Pic web app and contains over half-a-million examples of human preferences over model-generated images.
This dataset with URLs instead of the actual images (which makes it much smaller in size) can be found here.


See the corresponding paper Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation for more details.


If you want to download this dataset with URLs instead of images to save space, please see this version of the dataset. ### Supported Tasks and Leaderboards


Task: Select preferred image in test-set. ### Data Splits


The dataset has three main splits: train, validation, validation\_unique (with one example per prompt), test, and test\_unique.


If you find this work useful, please cite: ### LICENSE


MIT License


Copyright (c) 2021


Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:


The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.


THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
TAGS
#arxiv-2305.01569 #arxiv-2303.14420 #arxiv-2304.05977 #arxiv-2210.03927 #arxiv-2210.08402 #region-us 
 ### Dataset Summary


The Pick-a-Pic dataset was collected with the Pick-a-Pic web app and contains over half-a-million examples of human preferences over model-generated images.
This dataset with URLs instead of the actual images (which makes it much smaller in size) can be found here.


See the corresponding paper Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation for more details.


If you want to download this dataset with URLs instead of images to save space, please see this version of the dataset. ### Supported Tasks and Leaderboards


Task: Select preferred image in test-set. ### Data Splits


The dataset has three main splits: train, validation, validation\_unique (with one example per prompt), test, and test\_unique.


If you find this work useful, please cite: ### LICENSE


MIT License


Copyright (c) 2021


Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:


The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.


THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
passage: TAGS
#arxiv-2305.01569 #arxiv-2303.14420 #arxiv-2304.05977 #arxiv-2210.03927 #arxiv-2210.08402 #region-us 
### Dataset Summary


The Pick-a-Pic dataset was collected with the Pick-a-Pic web app and contains over half-a-million examples of human preferences over model-generated images.
This dataset with URLs instead of the actual images (which makes it much smaller in size) can be found here.


See the corresponding paper Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation for more details.


If you want to download this dataset with URLs instead of images to save space, please see this version of the dataset.### Supported Tasks and Leaderboards


Task: Select preferred image in test-set.### Data Splits


The dataset has three main splits: train, validation, validation\_unique (with one example per prompt), test, and test\_unique.


If you find this work useful, please cite: