
# Dataset Card for FrenchMedMCQA : A French Multiple-Choice Question Answering Corpus for Medical domain

## Table of Contents
- [Dataset Card for FrenchMedMCQA : A French Multiple-Choice Question Answering Corpus for Medical domain](#dataset-card-for-frenchmedmcqa--a-french-multiple-choice-question-answering-corpus-for-medical-domain)
  - [Table of Contents](#table-of-contents)
  - [Dataset Description](#dataset-description)
    - [Dataset Summary](#dataset-summary)
    - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
    - [Languages](#languages)
  - [Dataset Structure](#dataset-structure)
    - [Data Instances](#data-instances)
    - [Data Fields](#data-fields)
    - [Data Splits](#data-splits)
  - [Dataset Creation](#dataset-creation)
    - [Source Data](#source-data)
      - [Initial Data Collection and Normalization](#initial-data-collection-and-normalization)
    - [Personal and Sensitive Information](#personal-and-sensitive-information)
  - [Additional Information](#additional-information)
    - [Dataset Curators](#dataset-curators)
    - [Licensing Information](#licensing-information)
    - [Citation Information](#citation-information)
    - [Contact](#contact)

## Dataset Description

- **Homepage:** https://deft2023.univ-avignon.fr/
- **Repository:** https://deft2023.univ-avignon.fr/
- **Paper:** [FrenchMedMCQA: A French Multiple-Choice Question Answering Dataset for Medical domain](https://hal.science/hal-03824241/document)
- **Leaderboard:** Coming soon
- **Point of Contact:** [Yanis LABRAK](mailto:yanis.labrak@univ-avignon.fr)

### Dataset Summary

This paper introduces FrenchMedMCQA, the first publicly available Multiple-Choice Question Answering (MCQA) dataset in French for medical domain. It is composed of 3,105 questions taken from real exams of the French medical specialization diploma in pharmacy, mixing  single  and multiple answers.

Each instance of the dataset contains an identifier, a question, five possible answers and their manual correction(s).

We also propose first baseline models to automatically process this MCQA task in order to report on the current performances and to highlight the difficulty of the task. A detailed analysis of the results showed that it is necessary to have representations adapted to the medical domain or to the MCQA task: in our case, English specialized models yielded better results than generic French ones, even though FrenchMedMCQA is in French. Corpus, models and tools are available online.

### Supported Tasks and Leaderboards

Multiple-Choice Question Answering (MCQA)

### Languages

The questions and answers are available in French.

## Dataset Structure

### Data Instances

```json
{
  "id": "230bac49b0fe863b772410bc8d01a025f63c3c999065480131d6334abd2efeff",
  "prompt": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: We are giving you a scientific question (easy level) and five answers options (associated to « A », « B », « C », « D », « E »). Your task is to find the correct(s) answer(s) based on scientific facts, knowledge and reasoning. Don't generate anything other than one of the following characters : 'A B C D E'. ### Input: Parmi les affirmations suivantes, une seule est fausse, indiquer laquelle: les particules alpha (A) Sont formées de noyaux d'hélium (B) Sont peu pénétrantes (C) Toute l'énergie qu'elles transportent est cédée au long d'un parcours de quelques centimètres dans l'air (D) Sont arrêtées par une feuille de papier (E) Sont peu ionisantes ### Response: E",
  "prompt_no_answer": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: We are giving you a scientific question (easy level) and five answers options (associated to « A », « B », « C », « D », « E »). Your task is to find the correct(s) answer(s) based on scientific facts, knowledge and reasoning. Don't generate anything other than one of the following characters : 'A B C D E'. ### Input: Parmi les affirmations suivantes, une seule est fausse, indiquer laquelle: les particules alpha (A) Sont formées de noyaux d'hélium (B) Sont peu pénétrantes (C) Toute l'énergie qu'elles transportent est cédée au long d'un parcours de quelques centimètres dans l'air (D) Sont arrêtées par une feuille de papier (E) Sont peu ionisantes ### Response:",
  "correct_answers": [4],
}
```

### Data Fields

- `id` : a string question identifier for each example
- `prompt` : prompt text formatted for LLaMa (a string)
- `correct_answers` : Correct options, i.e., A, D and E

### Data Splits

| # Answers | Training | Validation | Test | Total |
|:---------:|:--------:|:----------:|:----:|:-----:|
|     1     |    595   |     164    |  321 | 1,080 |
|     2     |    528   |     45     |  97  |  670  |
|     3     |    718   |     71     |  141 |  930  |
|     4     |    296   |     30     |  56  |  382  |
|     5     |    34    |      2     |   7  |   43  |
|   Total   |   2171   |     312    |  622 | 3,105 |

## Dataset Creation

### Source Data

#### Initial Data Collection and Normalization

The questions and their associated candidate answer(s) were collected from real French pharmacy exams on the remede website. Questions and answers were manually created by medical experts and used during  examinations. The dataset is composed of 2,025 questions with multiple answers and 1,080 with a single one, for a total of 3,105 questions. Each instance of the dataset contains an identifier, a question, five options (labeled from A to E) and correct answer(s). The average question length is 14.17 tokens and the average answer length is 6.44 tokens. The vocabulary size is of 13k words, of which 3.8k are estimated medical domain-specific words (i.e. a word related to the medical field). We find an average of 2.49 medical domain-specific words in each question (17 % of the words) and 2 in each answer (36 % of the words). On average, a medical domain-specific word is present in 2 questions and in 8 answers.

### Personal and Sensitive Information

The corpora is free of personal or sensitive information.

## Additional Information

### Dataset Curators

The dataset was created by Labrak Yanis and Bazoge Adrien and Dufour Richard and Daille Béatrice and Gourraud Pierre-Antoine and Morin Emmanuel and Rouvier Mickael.

### Licensing Information

Apache 2.0

### Citation Information

If you find this useful in your research, please consider citing the dataset paper :

```latex
@inproceedings{labrak-etal-2022-frenchmedmcqa,
    title = "{F}rench{M}ed{MCQA}: A {F}rench Multiple-Choice Question Answering Dataset for Medical domain",
    author = "Labrak, Yanis  and
      Bazoge, Adrien  and
      Dufour, Richard  and
      Daille, Beatrice  and
      Gourraud, Pierre-Antoine  and
      Morin, Emmanuel  and
      Rouvier, Mickael",
    booktitle = "Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI)",
    month = dec,
    year = "2022",
    address = "Abu Dhabi, United Arab Emirates (Hybrid)",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.louhi-1.5",
    pages = "41--46",
    abstract = "This paper introduces FrenchMedMCQA, the first publicly available Multiple-Choice Question Answering (MCQA) dataset in French for medical domain. It is composed of 3,105 questions taken from real exams of the French medical specialization diploma in pharmacy, mixing single and multiple answers. Each instance of the dataset contains an identifier, a question, five possible answers and their manual correction(s). We also propose first baseline models to automatically process this MCQA task in order to report on the current performances and to highlight the difficulty of the task. A detailed analysis of the results showed that it is necessary to have representations adapted to the medical domain or to the MCQA task: in our case, English specialized models yielded better results than generic French ones, even though FrenchMedMCQA is in French. Corpus, models and tools are available online.",
}
```

### Contact

Thanks to contact [Yanis LABRAK](https://github.com/qanastek) for more information about this dataset.

task_categories:question-answering task_categories:multiple-choice task_ids:multiple-choice-qa task_ids:open-domain-qa annotations_creators:no-annotation language_creators:expert-generated multilinguality:monolingual size_categories:1k<n<10k source_datasets:original language:fr license:apache-2.0 region:us
{"annotations_creators": ["no-annotation"], "language_creators": ["expert-generated"], "language": ["fr"], "license": ["apache-2.0"], "multilinguality": ["monolingual"], "size_categories": ["1k<n<10k"], "source_datasets": ["original"], "task_categories": ["question-answering", "multiple-choice"], "task_ids": ["multiple-choice-qa", "open-domain-qa"], "paperswithcode_id": "frenchmedmcqa", "pretty_name": "FrenchMedMCQA"}
TAGS
#task_categories-question-answering #task_categories-multiple-choice #task_ids-multiple-choice-qa #task_ids-open-domain-qa #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-1k<n<10k #source_datasets-original #language-French #license-apache-2.0 #region-us 

Dataset Card for FrenchMedMCQA : A French Multiple-Choice Question Answering Corpus for Medical domain
======================================================================================================


Table of Contents
-----------------


* Dataset Card for FrenchMedMCQA : A French Multiple-Choice Question Answering Corpus for Medical domain
	+ Table of Contents
	+ Dataset Description
		- Dataset Summary
		- Supported Tasks and Leaderboards
		- Languages
	+ Dataset Structure
		- Data Instances
		- Data Fields
		- Data Splits
	+ Dataset Creation
		- Source Data
			* Initial Data Collection and Normalization
		- Personal and Sensitive Information
	+ Additional Information
		- Dataset Curators
		- Licensing Information
		- Citation Information
		- Contact


Dataset Description
-------------------


* Homepage: URL
* Repository: URL
* Paper: FrenchMedMCQA: A French Multiple-Choice Question Answering Dataset for Medical domain
* Leaderboard: Coming soon
* Point of Contact: Yanis LABRAK


### Dataset Summary


This paper introduces FrenchMedMCQA, the first publicly available Multiple-Choice Question Answering (MCQA) dataset in French for medical domain. It is composed of 3,105 questions taken from real exams of the French medical specialization diploma in pharmacy, mixing single and multiple answers.


Each instance of the dataset contains an identifier, a question, five possible answers and their manual correction(s).


We also propose first baseline models to automatically process this MCQA task in order to report on the current performances and to highlight the difficulty of the task. A detailed analysis of the results showed that it is necessary to have representations adapted to the medical domain or to the MCQA task: in our case, English specialized models yielded better results than generic French ones, even though FrenchMedMCQA is in French. Corpus, models and tools are available online.


### Supported Tasks and Leaderboards


Multiple-Choice Question Answering (MCQA)


### Languages


The questions and answers are available in French.


Dataset Structure
-----------------


### Data Instances


### Data Fields


* 'id' : a string question identifier for each example
* 'prompt' : prompt text formatted for LLaMa (a string)
* 'correct\_answers' : Correct options, i.e., A, D and E


### Data Splits



Dataset Creation
----------------


### Source Data


#### Initial Data Collection and Normalization


The questions and their associated candidate answer(s) were collected from real French pharmacy exams on the remede website. Questions and answers were manually created by medical experts and used during examinations. The dataset is composed of 2,025 questions with multiple answers and 1,080 with a single one, for a total of 3,105 questions. Each instance of the dataset contains an identifier, a question, five options (labeled from A to E) and correct answer(s). The average question length is 14.17 tokens and the average answer length is 6.44 tokens. The vocabulary size is of 13k words, of which 3.8k are estimated medical domain-specific words (i.e. a word related to the medical field). We find an average of 2.49 medical domain-specific words in each question (17 % of the words) and 2 in each answer (36 % of the words). On average, a medical domain-specific word is present in 2 questions and in 8 answers.


### Personal and Sensitive Information


The corpora is free of personal or sensitive information.


Additional Information
----------------------


### Dataset Curators


The dataset was created by Labrak Yanis and Bazoge Adrien and Dufour Richard and Daille Béatrice and Gourraud Pierre-Antoine and Morin Emmanuel and Rouvier Mickael.


### Licensing Information


Apache 2.0


If you find this useful in your research, please consider citing the dataset paper :


### Contact


Thanks to contact Yanis LABRAK for more information about this dataset.



### Dataset Summary


This paper introduces FrenchMedMCQA, the first publicly available Multiple-Choice Question Answering (MCQA) dataset in French for medical domain. It is composed of 3,105 questions taken from real exams of the French medical specialization diploma in pharmacy, mixing single and multiple answers.


Each instance of the dataset contains an identifier, a question, five possible answers and their manual correction(s).


We also propose first baseline models to automatically process this MCQA task in order to report on the current performances and to highlight the difficulty of the task. A detailed analysis of the results showed that it is necessary to have representations adapted to the medical domain or to the MCQA task: in our case, English specialized models yielded better results than generic French ones, even though FrenchMedMCQA is in French. Corpus, models and tools are available online. ### Supported Tasks and Leaderboards


Multiple-Choice Question Answering (MCQA) ### Languages


The questions and answers are available in French.


Dataset Structure
----------------- ### Data Instances ### Data Fields


* 'id' : a string question identifier for each example
* 'prompt' : prompt text formatted for LLaMa (a string)
* 'correct\_answers' : Correct options, i.e., A, D and E ### Data Splits



Dataset Creation
---------------- ### Source Data #### Initial Data Collection and Normalization


The questions and their associated candidate answer(s) were collected from real French pharmacy exams on the remede website. Questions and answers were manually created by medical experts and used during examinations. The dataset is composed of 2,025 questions with multiple answers and 1,080 with a single one, for a total of 3,105 questions. Each instance of the dataset contains an identifier, a question, five options (labeled from A to E) and correct answer(s). The average question length is 14.17 tokens and the average answer length is 6.44 tokens. The vocabulary size is of 13k words, of which 3.8k are estimated medical domain-specific words (i.e. a word related to the medical field). We find an average of 2.49 medical domain-specific words in each question (17 % of the words) and 2 in each answer (36 % of the words). On average, a medical domain-specific word is present in 2 questions and in 8 answers. ### Personal and Sensitive Information


The corpora is free of personal or sensitive information.


Additional Information
---------------------- ### Dataset Curators


The dataset was created by Labrak Yanis and Bazoge Adrien and Dufour Richard and Daille Béatrice and Gourraud Pierre-Antoine and Morin Emmanuel and Rouvier Mickael. ### Licensing Information


Apache 2.0


If you find this useful in your research, please consider citing the dataset paper : ### Contact


Thanks to contact Yanis LABRAK for more information about this dataset.
TAGS
#task_categories-question-answering #task_categories-multiple-choice #task_ids-multiple-choice-qa #task_ids-open-domain-qa #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-1k<n<10k #source_datasets-original #language-French #license-apache-2.0 #region-us 
 ### Dataset Summary


This paper introduces FrenchMedMCQA, the first publicly available Multiple-Choice Question Answering (MCQA) dataset in French for medical domain. It is composed of 3,105 questions taken from real exams of the French medical specialization diploma in pharmacy, mixing single and multiple answers.


Each instance of the dataset contains an identifier, a question, five possible answers and their manual correction(s).


We also propose first baseline models to automatically process this MCQA task in order to report on the current performances and to highlight the difficulty of the task. A detailed analysis of the results showed that it is necessary to have representations adapted to the medical domain or to the MCQA task: in our case, English specialized models yielded better results than generic French ones, even though FrenchMedMCQA is in French. Corpus, models and tools are available online. ### Supported Tasks and Leaderboards


Multiple-Choice Question Answering (MCQA) ### Languages


The questions and answers are available in French.


Dataset Structure
----------------- ### Data Instances ### Data Fields


* 'id' : a string question identifier for each example
* 'prompt' : prompt text formatted for LLaMa (a string)
* 'correct\_answers' : Correct options, i.e., A, D and E ### Data Splits



Dataset Creation
---------------- ### Source Data #### Initial Data Collection and Normalization


The questions and their associated candidate answer(s) were collected from real French pharmacy exams on the remede website. Questions and answers were manually created by medical experts and used during examinations. The dataset is composed of 2,025 questions with multiple answers and 1,080 with a single one, for a total of 3,105 questions. Each instance of the dataset contains an identifier, a question, five options (labeled from A to E) and correct answer(s). The average question length is 14.17 tokens and the average answer length is 6.44 tokens. The vocabulary size is of 13k words, of which 3.8k are estimated medical domain-specific words (i.e. a word related to the medical field). We find an average of 2.49 medical domain-specific words in each question (17 % of the words) and 2 in each answer (36 % of the words). On average, a medical domain-specific word is present in 2 questions and in 8 answers. ### Personal and Sensitive Information


The corpora is free of personal or sensitive information.


Additional Information
---------------------- ### Dataset Curators


The dataset was created by Labrak Yanis and Bazoge Adrien and Dufour Richard and Daille Béatrice and Gourraud Pierre-Antoine and Morin Emmanuel and Rouvier Mickael. ### Licensing Information


Apache 2.0


If you find this useful in your research, please consider citing the dataset paper : ### Contact


Thanks to contact Yanis LABRAK for more information about this dataset.
passage: TAGS
#task_categories-question-answering #task_categories-multiple-choice #task_ids-multiple-choice-qa #task_ids-open-domain-qa #annotations_creators-no-annotation #language_creators-expert-generated #multilinguality-monolingual #size_categories-1k<n<10k #source_datasets-original #language-French #license-apache-2.0 #region-us 
### Dataset Summary


This paper introduces FrenchMedMCQA, the first publicly available Multiple-Choice Question Answering (MCQA) dataset in French for medical domain. It is composed of 3,105 questions taken from real exams of the French medical specialization diploma in pharmacy, mixing single and multiple answers.


Each instance of the dataset contains an identifier, a question, five possible answers and their manual correction(s).


We also propose first baseline models to automatically process this MCQA task in order to report on the current performances and to highlight the difficulty of the task. A detailed analysis of the results showed that it is necessary to have representations adapted to the medical domain or to the MCQA task: in our case, English specialized models yielded better results than generic French ones, even though FrenchMedMCQA is in French. Corpus, models and tools are available online.### Supported Tasks and Leaderboards


Multiple-Choice Question Answering (MCQA)### Languages


The questions and answers are available in French.


Dataset Structure
-----------------### Data Instances### Data Fields


* 'id' : a string question identifier for each example
* 'prompt' : prompt text formatted for LLaMa (a string)
* 'correct\_answers' : Correct options, i.e., A, D and E### Data Splits



Dataset Creation
----------------### Source Data