
# Dataset Card for HEAD-QA

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** [HEAD-QA homepage](https://aghie.github.io/head-qa/)
- **Repository:** [HEAD-QA repository](https://github.com/aghie/head-qa)
- **Paper:** [HEAD-QA: A Healthcare Dataset for Complex Reasoning](https://www.aclweb.org/anthology/P19-1092/)
- **Leaderboard:** [HEAD-QA leaderboard](https://aghie.github.io/head-qa/#leaderboard-general)
- **Point of Contact:** [María Grandury](mailto:mariagrandury@gmail.com) (Dataset Submitter)

### Dataset Summary

HEAD-QA is a multi-choice HEAlthcare Dataset. The questions come from exams to access a specialized position in the 
Spanish healthcare system, and are challenging even for highly specialized humans. They are designed by the 
[Ministerio de Sanidad, Consumo y Bienestar Social](https://www.mscbs.gob.es/), who also provides direct 
[access](https://fse.mscbs.gob.es/fseweb/view/public/datosanteriores/cuadernosExamen/busquedaConvocatoria.xhtml)
to the exams of the last 5 years (in Spanish).

```
Date of the last update of the documents object of the reuse: January, 14th, 2019.
```

HEAD-QA tries to make these questions accesible for the Natural Language Processing community. We hope it is an useful resource towards achieving better QA systems. The dataset contains questions about the following topics:
- Medicine
- Nursing
- Psychology
- Chemistry
- Pharmacology
- Biology

### Supported Tasks and Leaderboards

- `multiple-choice-qa`: HEAD-QA is a multi-choice question answering testbed to encourage research on complex reasoning.

### Languages

The questions and answers are available in both Spanish (BCP-47 code: 'es-ES') and English (BCP-47 code: 'en').

The language by default is Spanish:
```
from datasets import load_dataset

data_es = load_dataset('head_qa')

data_en = load_dataset('head_qa', 'en')
```

## Dataset Structure

### Data Instances

A typical data point comprises a question `qtext`, multiple possible answers `atext` and the right answer `ra`.

An example from the HEAD-QA dataset looks as follows:
```
{
'qid': '1', 
'category': 'biology', 
'qtext': 'Los potenciales postsinápticos excitadores:',
'answers': [
	{
		'aid': 1, 
		'atext': 'Son de tipo todo o nada.'
	}, 
	{
		'aid': 2, 
		'atext': 'Son hiperpolarizantes.'
	},
	{
		'aid': 3, 
		'atext': 'Se pueden sumar.'
	},
	{
		'aid': 4, 
		'atext': 'Se propagan a largas distancias.'
	},
	{
		'aid': 5, 
		'atext': 'Presentan un periodo refractario.'
	}],
'ra': '3',
'image': <PIL.PngImagePlugin.PngImageFile image mode=RGB size=675x538 at 0x1B42B6A1668>,
'name': 'Cuaderno_2013_1_B',
'year': '2013'
}
```

### Data Fields

- `qid`: question identifier (int)
- `category`: category of the question: "medicine", "nursing", "psychology", "chemistry", "pharmacology", "biology"
- `qtext`: question text
- `answers`: list of possible answers. Each element of the list is a dictionary with 2 keys:
    - `aid`: answer identifier (int)
    - `atext`: answer text
- `ra`: `aid` of the right answer (int)
- `image`: (optional) a `PIL.Image.Image` object containing the image. Note that when accessing the image column: `dataset[0]["image"]` the image file is automatically decoded. Decoding of a large number of image files might take a significant amount of time. Thus it is important to first query the sample index before the `"image"` column, *i.e.* `dataset[0]["image"]` should **always** be preferred over `dataset["image"][0]`
- `name`: name of the exam from which the question was extracted
- `year`: year in which the exam took place

### Data Splits

The data is split into train, validation and test set for each of the two languages. The split sizes are as follow:

|         | Train  | Val   | Test |
| -----   | ------ | ----- | ---- |
| Spanish | 2657   | 1366  | 2742 |
| English | 2657   | 1366  | 2742 |

## Dataset Creation

### Curation Rationale

As motivation for the creation of this dataset, here is the abstract of the paper:

"We present HEAD-QA, a multi-choice question answering testbed to encourage research on complex reasoning. The questions
come from exams to access a specialized position in the Spanish healthcare system, and are challenging even for highly 
specialized humans. We then consider monolingual (Spanish) and cross-lingual (to English) experiments with information 
retrieval and neural techniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the results lag well 
behind human performance, demonstrating its usefulness as a benchmark for future work."

### Source Data

#### Initial Data Collection and Normalization

The questions come from exams to access a specialized position in the Spanish healthcare system, and are designed by the
[Ministerio de Sanidad, Consumo y Bienestar Social](https://www.mscbs.gob.es/), who also provides direct 
[access](https://fse.mscbs.gob.es/fseweb/view/public/datosanteriores/cuadernosExamen/busquedaConvocatoria.xhtml)
to the exams of the last 5 years (in Spanish).

#### Who are the source language producers?

The dataset was created by David Vilares and Carlos Gómez-Rodríguez.

### Annotations

The dataset does not contain any additional annotations.

#### Annotation process

[N/A]

#### Who are the annotators?

[N/A]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

The dataset was created by David Vilares and Carlos Gómez-Rodríguez.

### Licensing Information

According to the [HEAD-QA homepage](https://aghie.github.io/head-qa/#legal-requirements):

The Ministerio de Sanidad, Consumo y Biniestar Social allows the redistribution of the exams and their content under [certain conditions:](https://www.mscbs.gob.es/avisoLegal/home.htm)

- The denaturalization of the content of the information is prohibited in any circumstance.
- The user is obliged to cite the source of the documents subject to reuse.
- The user is obliged to indicate the date of the last update of the documents object of the reuse.

According to the [HEAD-QA repository](https://github.com/aghie/head-qa/blob/master/LICENSE):

The dataset is licensed under the [MIT License](https://mit-license.org/).

### Citation Information

```
@inproceedings{vilares-gomez-rodriguez-2019-head,
    title = "{HEAD}-{QA}: A Healthcare Dataset for Complex Reasoning",
    author = "Vilares, David  and
      G{\'o}mez-Rodr{\'i}guez, Carlos",
    booktitle = "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2019",
    address = "Florence, Italy",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/P19-1092",
    doi = "10.18653/v1/P19-1092",
    pages = "960--966",
    abstract = "We present HEAD-QA, a multi-choice question answering testbed to encourage research on complex reasoning. The questions come from exams to access a specialized position in the Spanish healthcare system, and are challenging even for highly specialized humans. We then consider monolingual (Spanish) and cross-lingual (to English) experiments with information retrieval and neural techniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the results lag well behind human performance, demonstrating its usefulness as a benchmark for future work.",
}
```
### Contributions

Thanks to [@mariagrandury](https://github.com/mariagrandury) for adding this dataset.
task_categories:question-answering task_ids:multiple-choice-qa annotations_creators:no-annotation language_creators:expert-generated multilinguality:monolingual size_categories:1K<n<10K source_datasets:original language:en language:es license:mit region:us
{"annotations_creators": ["no-annotation"], "language_creators": ["expert-generated"], "language": ["en", "es"], "license": ["mit"], "multilinguality": ["monolingual"], "size_categories": ["1K<n<10K"], "source_datasets": ["original"], "task_categories": ["question-answering"], "task_ids": ["multiple-choice-qa"], "paperswithcode_id": "headqa", "pretty_name": "HEAD-QA", "config_names": ["en", "es"], "dataset_info": [{"config_name": "es", "features": [{"name": "name", "dtype": "string"}, {"name": "year", "dtype": "string"}, {"name": "category", "dtype": "string"}, {"name": "qid", "dtype": "int32"}, {"name": "qtext", "dtype": "string"}, {"name": "ra", "dtype": "int32"}, {"name": "image", "dtype": "image"}, {"name": "answers", "list": [{"name": "aid", "dtype": "int32"}, {"name": "atext", "dtype": "string"}]}], "splits": [{"name": "train", "num_bytes": 1229678, "num_examples": 2657}, {"name": "test", "num_bytes": 1204006, "num_examples": 2742}, {"name": "validation", "num_bytes": 573354, "num_examples": 1366}], "download_size": 79365502, "dataset_size": 3007038}, {"config_name": "en", "features": [{"name": "name", "dtype": "string"}, {"name": "year", "dtype": "string"}, {"name": "category", "dtype": "string"}, {"name": "qid", "dtype": "int32"}, {"name": "qtext", "dtype": "string"}, {"name": "ra", "dtype": "int32"}, {"name": "image", "dtype": "image"}, {"name": "answers", "list": [{"name": "aid", "dtype": "int32"}, {"name": "atext", "dtype": "string"}]}], "splits": [{"name": "train", "num_bytes": 1156808, "num_examples": 2657}, {"name": "test", "num_bytes": 1131536, "num_examples": 2742}, {"name": "validation", "num_bytes": 539892, "num_examples": 1366}], "download_size": 79365502, "dataset_size": 2828236}]}
TAGS
#task_categories-question-answering #task_ids-multiple-choice-qa #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #language-Spanish #license-mit #region-us 

Dataset Card for HEAD-QA
========================


Table of Contents
-----------------


* Dataset Description
	+ Dataset Summary
	+ Supported Tasks and Leaderboards
	+ Languages
* Dataset Structure
	+ Data Instances
	+ Data Fields
	+ Data Splits
* Dataset Creation
	+ Curation Rationale
	+ Source Data
	+ Annotations
	+ Personal and Sensitive Information
* Considerations for Using the Data
	+ Social Impact of Dataset
	+ Discussion of Biases
	+ Other Known Limitations
* Additional Information
	+ Dataset Curators
	+ Licensing Information
	+ Citation Information
	+ Contributions


Dataset Description
-------------------


* Homepage: HEAD-QA homepage
* Repository: HEAD-QA repository
* Paper: HEAD-QA: A Healthcare Dataset for Complex Reasoning
* Leaderboard: HEAD-QA leaderboard
* Point of Contact: María Grandury (Dataset Submitter)


### Dataset Summary


HEAD-QA is a multi-choice HEAlthcare Dataset. The questions come from exams to access a specialized position in the
Spanish healthcare system, and are challenging even for highly specialized humans. They are designed by the
Ministerio de Sanidad, Consumo y Bienestar Social, who also provides direct
access
to the exams of the last 5 years (in Spanish).


HEAD-QA tries to make these questions accesible for the Natural Language Processing community. We hope it is an useful resource towards achieving better QA systems. The dataset contains questions about the following topics:


* Medicine
* Nursing
* Psychology
* Chemistry
* Pharmacology
* Biology


### Supported Tasks and Leaderboards


* 'multiple-choice-qa': HEAD-QA is a multi-choice question answering testbed to encourage research on complex reasoning.


### Languages


The questions and answers are available in both Spanish (BCP-47 code: 'es-ES') and English (BCP-47 code: 'en').


The language by default is Spanish:


Dataset Structure
-----------------


### Data Instances


A typical data point comprises a question 'qtext', multiple possible answers 'atext' and the right answer 'ra'.


An example from the HEAD-QA dataset looks as follows:


### Data Fields


* 'qid': question identifier (int)
* 'category': category of the question: "medicine", "nursing", "psychology", "chemistry", "pharmacology", "biology"
* 'qtext': question text
* 'answers': list of possible answers. Each element of the list is a dictionary with 2 keys:
	+ 'aid': answer identifier (int)
	+ 'atext': answer text
* 'ra': 'aid' of the right answer (int)
* 'image': (optional) a 'PIL.Image.Image' object containing the image. Note that when accessing the image column: 'dataset[0]["image"]' the image file is automatically decoded. Decoding of a large number of image files might take a significant amount of time. Thus it is important to first query the sample index before the '"image"' column, *i.e.* 'dataset[0]["image"]' should always be preferred over 'dataset["image"][0]'
* 'name': name of the exam from which the question was extracted
* 'year': year in which the exam took place


### Data Splits


The data is split into train, validation and test set for each of the two languages. The split sizes are as follow:



Dataset Creation
----------------


### Curation Rationale


As motivation for the creation of this dataset, here is the abstract of the paper:


"We present HEAD-QA, a multi-choice question answering testbed to encourage research on complex reasoning. The questions
come from exams to access a specialized position in the Spanish healthcare system, and are challenging even for highly
specialized humans. We then consider monolingual (Spanish) and cross-lingual (to English) experiments with information
retrieval and neural techniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the results lag well
behind human performance, demonstrating its usefulness as a benchmark for future work."


### Source Data


#### Initial Data Collection and Normalization


The questions come from exams to access a specialized position in the Spanish healthcare system, and are designed by the
Ministerio de Sanidad, Consumo y Bienestar Social, who also provides direct
access
to the exams of the last 5 years (in Spanish).


#### Who are the source language producers?


The dataset was created by David Vilares and Carlos Gómez-Rodríguez.


### Annotations


The dataset does not contain any additional annotations.


#### Annotation process


[N/A]


#### Who are the annotators?


[N/A]


### Personal and Sensitive Information


Considerations for Using the Data
---------------------------------


### Social Impact of Dataset


### Discussion of Biases


### Other Known Limitations


Additional Information
----------------------


### Dataset Curators


The dataset was created by David Vilares and Carlos Gómez-Rodríguez.


### Licensing Information


According to the HEAD-QA homepage:


The Ministerio de Sanidad, Consumo y Biniestar Social allows the redistribution of the exams and their content under certain conditions:


* The denaturalization of the content of the information is prohibited in any circumstance.
* The user is obliged to cite the source of the documents subject to reuse.
* The user is obliged to indicate the date of the last update of the documents object of the reuse.


According to the HEAD-QA repository:


The dataset is licensed under the MIT License.


### Contributions


Thanks to @mariagrandury for adding this dataset.



### Dataset Summary


HEAD-QA is a multi-choice HEAlthcare Dataset. The questions come from exams to access a specialized position in the
Spanish healthcare system, and are challenging even for highly specialized humans. They are designed by the
Ministerio de Sanidad, Consumo y Bienestar Social, who also provides direct
access
to the exams of the last 5 years (in Spanish).


HEAD-QA tries to make these questions accesible for the Natural Language Processing community. We hope it is an useful resource towards achieving better QA systems. The dataset contains questions about the following topics:


* Medicine
* Nursing
* Psychology
* Chemistry
* Pharmacology
* Biology ### Supported Tasks and Leaderboards


* 'multiple-choice-qa': HEAD-QA is a multi-choice question answering testbed to encourage research on complex reasoning. ### Languages


The questions and answers are available in both Spanish (BCP-47 code: 'es-ES') and English (BCP-47 code: 'en').


The language by default is Spanish:


Dataset Structure
----------------- ### Data Instances


A typical data point comprises a question 'qtext', multiple possible answers 'atext' and the right answer 'ra'.


An example from the HEAD-QA dataset looks as follows: ### Data Fields


* 'qid': question identifier (int)
* 'category': category of the question: "medicine", "nursing", "psychology", "chemistry", "pharmacology", "biology"
* 'qtext': question text
* 'answers': list of possible answers. Each element of the list is a dictionary with 2 keys:
	+ 'aid': answer identifier (int)
	+ 'atext': answer text
* 'ra': 'aid' of the right answer (int)
* 'image': (optional) a 'PIL.Image.Image' object containing the image. Note that when accessing the image column: 'dataset[0]["image"]' the image file is automatically decoded. Decoding of a large number of image files might take a significant amount of time. Thus it is important to first query the sample index before the '"image"' column, *i.e.* 'dataset[0]["image"]' should always be preferred over 'dataset["image"][0]'
* 'name': name of the exam from which the question was extracted
* 'year': year in which the exam took place ### Data Splits


The data is split into train, validation and test set for each of the two languages. The split sizes are as follow:



Dataset Creation
---------------- ### Curation Rationale


As motivation for the creation of this dataset, here is the abstract of the paper:


"We present HEAD-QA, a multi-choice question answering testbed to encourage research on complex reasoning. The questions
come from exams to access a specialized position in the Spanish healthcare system, and are challenging even for highly
specialized humans. We then consider monolingual (Spanish) and cross-lingual (to English) experiments with information
retrieval and neural techniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the results lag well
behind human performance, demonstrating its usefulness as a benchmark for future work." ### Source Data #### Initial Data Collection and Normalization


The questions come from exams to access a specialized position in the Spanish healthcare system, and are designed by the
Ministerio de Sanidad, Consumo y Bienestar Social, who also provides direct
access
to the exams of the last 5 years (in Spanish). #### Who are the source language producers?


The dataset was created by David Vilares and Carlos Gómez-Rodríguez. ### Annotations


The dataset does not contain any additional annotations. #### Annotation process


[N/A] #### Who are the annotators?


[N/A] ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators


The dataset was created by David Vilares and Carlos Gómez-Rodríguez. ### Licensing Information


According to the HEAD-QA homepage:


The Ministerio de Sanidad, Consumo y Biniestar Social allows the redistribution of the exams and their content under certain conditions:


* The denaturalization of the content of the information is prohibited in any circumstance.
* The user is obliged to cite the source of the documents subject to reuse.
* The user is obliged to indicate the date of the last update of the documents object of the reuse.


According to the HEAD-QA repository:


The dataset is licensed under the MIT License. ### Contributions


Thanks to @mariagrandury for adding this dataset.
TAGS
#task_categories-question-answering #task_ids-multiple-choice-qa #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #language-Spanish #license-mit #region-us 
 ### Dataset Summary


HEAD-QA is a multi-choice HEAlthcare Dataset. The questions come from exams to access a specialized position in the
Spanish healthcare system, and are challenging even for highly specialized humans. They are designed by the
Ministerio de Sanidad, Consumo y Bienestar Social, who also provides direct
access
to the exams of the last 5 years (in Spanish).


HEAD-QA tries to make these questions accesible for the Natural Language Processing community. We hope it is an useful resource towards achieving better QA systems. The dataset contains questions about the following topics:


* Medicine
* Nursing
* Psychology
* Chemistry
* Pharmacology
* Biology ### Supported Tasks and Leaderboards


* 'multiple-choice-qa': HEAD-QA is a multi-choice question answering testbed to encourage research on complex reasoning. ### Languages


The questions and answers are available in both Spanish (BCP-47 code: 'es-ES') and English (BCP-47 code: 'en').


The language by default is Spanish:


Dataset Structure
----------------- ### Data Instances


A typical data point comprises a question 'qtext', multiple possible answers 'atext' and the right answer 'ra'.


An example from the HEAD-QA dataset looks as follows: ### Data Fields


* 'qid': question identifier (int)
* 'category': category of the question: "medicine", "nursing", "psychology", "chemistry", "pharmacology", "biology"
* 'qtext': question text
* 'answers': list of possible answers. Each element of the list is a dictionary with 2 keys:
	+ 'aid': answer identifier (int)
	+ 'atext': answer text
* 'ra': 'aid' of the right answer (int)
* 'image': (optional) a 'PIL.Image.Image' object containing the image. Note that when accessing the image column: 'dataset[0]["image"]' the image file is automatically decoded. Decoding of a large number of image files might take a significant amount of time. Thus it is important to first query the sample index before the '"image"' column, *i.e.* 'dataset[0]["image"]' should always be preferred over 'dataset["image"][0]'
* 'name': name of the exam from which the question was extracted
* 'year': year in which the exam took place ### Data Splits


The data is split into train, validation and test set for each of the two languages. The split sizes are as follow:



Dataset Creation
---------------- ### Curation Rationale


As motivation for the creation of this dataset, here is the abstract of the paper:


"We present HEAD-QA, a multi-choice question answering testbed to encourage research on complex reasoning. The questions
come from exams to access a specialized position in the Spanish healthcare system, and are challenging even for highly
specialized humans. We then consider monolingual (Spanish) and cross-lingual (to English) experiments with information
retrieval and neural techniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the results lag well
behind human performance, demonstrating its usefulness as a benchmark for future work." ### Source Data #### Initial Data Collection and Normalization


The questions come from exams to access a specialized position in the Spanish healthcare system, and are designed by the
Ministerio de Sanidad, Consumo y Bienestar Social, who also provides direct
access
to the exams of the last 5 years (in Spanish). #### Who are the source language producers?


The dataset was created by David Vilares and Carlos Gómez-Rodríguez. ### Annotations


The dataset does not contain any additional annotations. #### Annotation process


[N/A] #### Who are the annotators?


[N/A] ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators


The dataset was created by David Vilares and Carlos Gómez-Rodríguez. ### Licensing Information


According to the HEAD-QA homepage:


The Ministerio de Sanidad, Consumo y Biniestar Social allows the redistribution of the exams and their content under certain conditions:


* The denaturalization of the content of the information is prohibited in any circumstance.
* The user is obliged to cite the source of the documents subject to reuse.
* The user is obliged to indicate the date of the last update of the documents object of the reuse.


According to the HEAD-QA repository:


The dataset is licensed under the MIT License. ### Contributions


Thanks to @mariagrandury for adding this dataset.
passage: TAGS
#task_categories-question-answering #task_ids-multiple-choice-qa #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-1K<n<10K #source_datasets-original #language-English #language-Spanish #license-mit #region-us 
### Dataset Summary


HEAD-QA is a multi-choice HEAlthcare Dataset. The questions come from exams to access a specialized position in the
Spanish healthcare system, and are challenging even for highly specialized humans. They are designed by the
Ministerio de Sanidad, Consumo y Bienestar Social, who also provides direct
access
to the exams of the last 5 years (in Spanish).


HEAD-QA tries to make these questions accesible for the Natural Language Processing community. We hope it is an useful resource towards achieving better QA systems. The dataset contains questions about the following topics:


* Medicine
* Nursing
* Psychology
* Chemistry
* Pharmacology
* Biology### Supported Tasks and Leaderboards


* 'multiple-choice-qa': HEAD-QA is a multi-choice question answering testbed to encourage research on complex reasoning.### Languages


The questions and answers are available in both Spanish (BCP-47 code: 'es-ES') and English (BCP-47 code: 'en').


The language by default is Spanish:


Dataset Structure
-----------------### Data Instances


A typical data point comprises a question 'qtext', multiple possible answers 'atext' and the right answer 'ra'.


An example from the HEAD-QA dataset looks as follows: passage: ### Data Fields


* 'qid': question identifier (int)
* 'category': category of the question: "medicine", "nursing", "psychology", "chemistry", "pharmacology", "biology"
* 'qtext': question text
* 'answers': list of possible answers. Each element of the list is a dictionary with 2 keys:
	+ 'aid': answer identifier (int)
	+ 'atext': answer text
* 'ra': 'aid' of the right answer (int)
* 'image': (optional) a 'PIL.Image.Image' object containing the image. Note that when accessing the image column: 'dataset[0]["image"]' the image file is automatically decoded. Decoding of a large number of image files might take a significant amount of time. Thus it is important to first query the sample index before the '"image"' column, *i.e.* 'dataset[0]["image"]' should always be preferred over 'dataset["image"][0]'
* 'name': name of the exam from which the question was extracted
* 'year': year in which the exam took place### Data Splits


The data is split into train, validation and test set for each of the two languages. The split sizes are as follow:



Dataset Creation
----------------### Curation Rationale


As motivation for the creation of this dataset, here is the abstract of the paper:


"We present HEAD-QA, a multi-choice question answering testbed to encourage research on complex reasoning. The questions
come from exams to access a specialized position in the Spanish healthcare system, and are challenging even for highly
specialized humans. We then consider monolingual (Spanish) and cross-lingual (to English) experiments with information
retrieval and neural techniques. We show that: (i) HEAD-QA challenges current methods, and (ii) the results lag well
behind human performance, demonstrating its usefulness as a benchmark for future work."### Source Data#### Initial Data Collection and Normalization


The questions come from exams to access a specialized position in the Spanish healthcare system, and are designed by the
Ministerio de Sanidad, Consumo y Bienestar Social, who also provides direct
access
to the exams of the last 5 years (in Spanish).#### Who are the source language producers?


The dataset was created by David Vilares and Carlos Gómez-Rodríguez.### Annotations


The dataset does not contain any additional annotations.#### Annotation process


[N/A]#### Who are the annotators?


[N/A]### Personal and Sensitive Information


Considerations for Using the Data
---------------------------------### Social Impact of Dataset### Discussion of Biases### Other Known Limitations


Additional Information
----------------------### Dataset Curators


The dataset was created by David Vilares and Carlos Gómez-Rodríguez.