
# Dataset Card for WSDMCup2023

## Dataset Description

- **Homepage:** [Toloka Visual Question Answering Challenge](https://toloka.ai/challenges/wsdm2023)
- **Repository:** [WSDM Cup 2023 Starter Pack](https://github.com/Toloka/WSDMCup2023)
- **Paper:** <https://arxiv.org/abs/2309.16511>
- **Leaderboard:** [CodaLab Competition Leaderboard](https://codalab.lisn.upsaclay.fr/competitions/7434#results)
- **Point of Contact:** research@toloka.ai

| Question | Image and Answer |
| --- | --- |
| What do you use to hit the ball? | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/KUsGAc_eqdMcNxkBXzzl/KUsGAc_eqdMcNxkBXzzl_webp_1280_x2.webp" width="228" alt="What do you use to hit the ball?"> |
| What do people use for cutting? | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/brXEVYckNLfQKcfNu4DF/brXEVYckNLfQKcfNu4DF_webp_1280_x2.webp" width="228" alt="What do people use for cutting?"> |
| What do we use to support the immune system and get vitamin C? | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/HQ0A-ZvZCGCmYfTs83K7/HQ0A-ZvZCGCmYfTs83K7_webp_1280_x2.webp" width="228" alt="What do we use to support the immune system and get vitamin C?"> |

### Dataset Summary

The WSDMCup2023 Dataset consists of images associated with textual questions.
One entry (instance) in our dataset is a question-image pair labeled with the ground truth coordinates of a bounding box containing
the visual answer to the given question. The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, [MS COCO](https://cocodataset.org/). All data labeling was performed on the [Toloka crowdsourcing platform](https://toloka.ai/).

Our dataset has 45,199 instances split among three subsets: train (38,990 instances), public test (1,705 instances),
and private test (4,504 instances). The entire train dataset was available for everyone since the start of the challenge.
The public test dataset was available since the evaluation phase of the competition but without any ground truth labels.
After the end of the competition, public and private sets were released.

## Dataset Citation

Please cite the challenge results or dataset description as follows.

- Ustalov D., Pavlichenko N., Koshelev S., Likhobaba D., and Smirnova A. [Toloka Visual Question Answering Benchmark](https://arxiv.org/abs/2309.16511). 2023. arXiv: [2309.16511 [cs.CV]](https://arxiv.org/abs/2309.16511).

```bibtex
@inproceedings{TolokaWSDMCup2023,
  author    = {Ustalov, Dmitry and Pavlichenko, Nikita and Koshelev, Sergey and Likhobaba, Daniil and Smirnova, Alisa},
  title     = {{Toloka Visual Question Answering Benchmark}},
  year      = {2023},
  eprint    = {2309.16511},
  eprinttype = {arxiv},
  eprintclass = {cs.CV},
  language  = {english},
}
```

### Supported Tasks and Leaderboards

Grounding Visual Question Answering

### Language

English

## Dataset Structure

### Data Instances
A data instance contains a URL to the picture, information about the image size - width and height, information about the ground truth bounding box - left top and right bottom dots, and contains the question related to the picture.
```
{'image': https://toloka-cdn.azureedge.net/wsdmcup2023/000000000013.jpg,
'width': 640,
'height': 427,
'left': 129,
'top': 192,
'right': 155,
'bottom': 212,
'question': What does it use to breath?}
```
### Data Fields

* image: contains URL to the image
* width: value in pixels of image width
* height: value in pixels of image height
* left: the x coordinate in pixels to determine the left-top dot of the bounding box
* top: the y coordinate in pixels to determine the left-top dot of the bounding box
* right: the x coordinate in pixels to determine the right-bottom dot of the bounding box
* bottom: the y coordinate in pixels to determine the right-bottom dot of the bounding box
* question: a question related to the picture

### Data Splits
There are four splits in the data: train, train_sample, test_public, and test_private. 'train' split contains the full pull for model training.
The 'train-sample' split contains the part of the 'train' split. The 'test_public' split contains public data to test the model. 
The 'test_private' split contains private data for the final model test.

### Source Data

The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, [MS COCO](https://cocodataset.org/).

### Annotations

All data labeling was performed on the [Toloka crowdsourcing platform](https://toloka.ai/).

Only annotators who self-reported the knowledge of English had access to the annotation task.

### Citation Information

* Competition: https://toloka.ai/challenges/wsdm2023
* CodaLab: https://codalab.lisn.upsaclay.fr/competitions/7434
* Dataset: https://doi.org/10.5281/zenodo.7057740
task_categories:visual-question-answering task_ids:visual-question-answering annotations_creators:crowdsourced language_creators:crowdsourced multilinguality:monolingual size_categories:10K<n<100K language:en license:cc-by-4.0 toloka arxiv:2309.16511 region:us
{"annotations_creators": ["crowdsourced"], "language_creators": ["crowdsourced"], "language": ["en"], "license": ["cc-by-4.0"], "multilinguality": ["monolingual"], "size_categories": ["10K<n<100K"], "source_datasets": [], "task_categories": ["visual-question-answering"], "task_ids": ["visual-question-answering"], "pretty_name": "WSDMCup2023", "tags": ["toloka"], "dataset_info": {"features": [{"name": "image", "dtype": "string"}, {"name": "width", "dtype": "int64"}, {"name": "height", "dtype": "int64"}, {"name": "left", "dtype": "int64"}, {"name": "top", "dtype": "int64"}, {"name": "right", "dtype": "int64"}, {"name": "bottom", "dtype": "int64"}, {"name": "question", "dtype": "string"}], "splits": [{"name": "train", "num_examples": 38990}, {"name": "train_sample", "num_examples": 1000}, {"name": "test_public", "num_examples": 1705}, {"name": "test_private", "num_examples": 4504}], "config_name": "wsdmcup2023"}}
TAGS
#task_categories-visual-question-answering #task_ids-visual-question-answering #annotations_creators-crowdsourced #language_creators-crowdsourced #multilinguality-monolingual #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #toloka #arxiv-2309.16511 #region-us 

Dataset Card for WSDMCup2023
============================


Dataset Description
-------------------


* Homepage: Toloka Visual Question Answering Challenge
* Repository: WSDM Cup 2023 Starter Pack
* Paper: <URL
* Leaderboard: CodaLab Competition Leaderboard
* Point of Contact: research@URL



### Dataset Summary


The WSDMCup2023 Dataset consists of images associated with textual questions.
One entry (instance) in our dataset is a question-image pair labeled with the ground truth coordinates of a bounding box containing
the visual answer to the given question. The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, MS COCO. All data labeling was performed on the Toloka crowdsourcing platform.


Our dataset has 45,199 instances split among three subsets: train (38,990 instances), public test (1,705 instances),
and private test (4,504 instances). The entire train dataset was available for everyone since the start of the challenge.
The public test dataset was available since the evaluation phase of the competition but without any ground truth labels.
After the end of the competition, public and private sets were released.


Dataset Citation
----------------


Please cite the challenge results or dataset description as follows.


* Ustalov D., Pavlichenko N., Koshelev S., Likhobaba D., and Smirnova A. Toloka Visual Question Answering Benchmark. 2023. arXiv: [2309.16511 [cs.CV]](URL


### Supported Tasks and Leaderboards


Grounding Visual Question Answering


### Language


English


Dataset Structure
-----------------


### Data Instances


A data instance contains a URL to the picture, information about the image size - width and height, information about the ground truth bounding box - left top and right bottom dots, and contains the question related to the picture.


### Data Fields


* image: contains URL to the image
* width: value in pixels of image width
* height: value in pixels of image height
* left: the x coordinate in pixels to determine the left-top dot of the bounding box
* top: the y coordinate in pixels to determine the left-top dot of the bounding box
* right: the x coordinate in pixels to determine the right-bottom dot of the bounding box
* bottom: the y coordinate in pixels to determine the right-bottom dot of the bounding box
* question: a question related to the picture


### Data Splits


There are four splits in the data: train, train\_sample, test\_public, and test\_private. 'train' split contains the full pull for model training.
The 'train-sample' split contains the part of the 'train' split. The 'test\_public' split contains public data to test the model.
The 'test\_private' split contains private data for the final model test.


### Source Data


The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, MS COCO.


### Annotations


All data labeling was performed on the Toloka crowdsourcing platform.


Only annotators who self-reported the knowledge of English had access to the annotation task.


* Competition: URL
* CodaLab: URL
* Dataset: URL



### Dataset Summary


The WSDMCup2023 Dataset consists of images associated with textual questions.
One entry (instance) in our dataset is a question-image pair labeled with the ground truth coordinates of a bounding box containing
the visual answer to the given question. The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, MS COCO. All data labeling was performed on the Toloka crowdsourcing platform.


Our dataset has 45,199 instances split among three subsets: train (38,990 instances), public test (1,705 instances),
and private test (4,504 instances). The entire train dataset was available for everyone since the start of the challenge.
The public test dataset was available since the evaluation phase of the competition but without any ground truth labels.
After the end of the competition, public and private sets were released.


Dataset Citation
----------------


Please cite the challenge results or dataset description as follows.


* Ustalov D., Pavlichenko N., Koshelev S., Likhobaba D., and Smirnova A. Toloka Visual Question Answering Benchmark. 2023. arXiv: [2309.16511 [cs.CV]](URL ### Supported Tasks and Leaderboards


Grounding Visual Question Answering ### Language


English


Dataset Structure
----------------- ### Data Instances


A data instance contains a URL to the picture, information about the image size - width and height, information about the ground truth bounding box - left top and right bottom dots, and contains the question related to the picture. ### Data Fields


* image: contains URL to the image
* width: value in pixels of image width
* height: value in pixels of image height
* left: the x coordinate in pixels to determine the left-top dot of the bounding box
* top: the y coordinate in pixels to determine the left-top dot of the bounding box
* right: the x coordinate in pixels to determine the right-bottom dot of the bounding box
* bottom: the y coordinate in pixels to determine the right-bottom dot of the bounding box
* question: a question related to the picture ### Data Splits


There are four splits in the data: train, train\_sample, test\_public, and test\_private. 'train' split contains the full pull for model training.
The 'train-sample' split contains the part of the 'train' split. The 'test\_public' split contains public data to test the model.
The 'test\_private' split contains private data for the final model test. ### Source Data


The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, MS COCO. ### Annotations


All data labeling was performed on the Toloka crowdsourcing platform.


Only annotators who self-reported the knowledge of English had access to the annotation task.


* Competition: URL
* CodaLab: URL
* Dataset: URL
TAGS
#task_categories-visual-question-answering #task_ids-visual-question-answering #annotations_creators-crowdsourced #language_creators-crowdsourced #multilinguality-monolingual #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #toloka #arxiv-2309.16511 #region-us 
 ### Dataset Summary


The WSDMCup2023 Dataset consists of images associated with textual questions.
One entry (instance) in our dataset is a question-image pair labeled with the ground truth coordinates of a bounding box containing
the visual answer to the given question. The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, MS COCO. All data labeling was performed on the Toloka crowdsourcing platform.


Our dataset has 45,199 instances split among three subsets: train (38,990 instances), public test (1,705 instances),
and private test (4,504 instances). The entire train dataset was available for everyone since the start of the challenge.
The public test dataset was available since the evaluation phase of the competition but without any ground truth labels.
After the end of the competition, public and private sets were released.


Dataset Citation
----------------


Please cite the challenge results or dataset description as follows.


* Ustalov D., Pavlichenko N., Koshelev S., Likhobaba D., and Smirnova A. Toloka Visual Question Answering Benchmark. 2023. arXiv: [2309.16511 [cs.CV]](URL ### Supported Tasks and Leaderboards


Grounding Visual Question Answering ### Language


English


Dataset Structure
----------------- ### Data Instances


A data instance contains a URL to the picture, information about the image size - width and height, information about the ground truth bounding box - left top and right bottom dots, and contains the question related to the picture. ### Data Fields


* image: contains URL to the image
* width: value in pixels of image width
* height: value in pixels of image height
* left: the x coordinate in pixels to determine the left-top dot of the bounding box
* top: the y coordinate in pixels to determine the left-top dot of the bounding box
* right: the x coordinate in pixels to determine the right-bottom dot of the bounding box
* bottom: the y coordinate in pixels to determine the right-bottom dot of the bounding box
* question: a question related to the picture ### Data Splits


There are four splits in the data: train, train\_sample, test\_public, and test\_private. 'train' split contains the full pull for model training.
The 'train-sample' split contains the part of the 'train' split. The 'test\_public' split contains public data to test the model.
The 'test\_private' split contains private data for the final model test. ### Source Data


The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, MS COCO. ### Annotations


All data labeling was performed on the Toloka crowdsourcing platform.


Only annotators who self-reported the knowledge of English had access to the annotation task.


* Competition: URL
* CodaLab: URL
* Dataset: URL
passage: TAGS
#task_categories-visual-question-answering #task_ids-visual-question-answering #annotations_creators-crowdsourced #language_creators-crowdsourced #multilinguality-monolingual #size_categories-10K<n<100K #language-English #license-cc-by-4.0 #toloka #arxiv-2309.16511 #region-us 
### Dataset Summary


The WSDMCup2023 Dataset consists of images associated with textual questions.
One entry (instance) in our dataset is a question-image pair labeled with the ground truth coordinates of a bounding box containing
the visual answer to the given question. The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in
Context dataset, MS COCO. All data labeling was performed on the Toloka crowdsourcing platform.


Our dataset has 45,199 instances split among three subsets: train (38,990 instances), public test (1,705 instances),
and private test (4,504 instances). The entire train dataset was available for everyone since the start of the challenge.
The public test dataset was available since the evaluation phase of the competition but without any ground truth labels.
After the end of the competition, public and private sets were released.


Dataset Citation
----------------


Please cite the challenge results or dataset description as follows.


* Ustalov D., Pavlichenko N., Koshelev S., Likhobaba D., and Smirnova A. Toloka Visual Question Answering Benchmark. 2023. arXiv: [2309.16511 [cs.CV]](URL### Supported Tasks and Leaderboards


Grounding Visual Question Answering### Language


English


Dataset Structure
-----------------### Data Instances


A data instance contains a URL to the picture, information about the image size - width and height, information about the ground truth bounding box - left top and right bottom dots, and contains the question related to the picture.