# Dataset Card for reasoning_bg

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** https://github.com/mhardalov/bg-reason-BERT
- **Repository:** https://github.com/mhardalov/bg-reason-BERT
- **Paper:** [Beyond English-Only Reading Comprehension: Experiments in Zero-Shot Multilingual Transfer for Bulgarian](https://arxiv.org/abs/1908.01519)
- **Leaderboard:** [N/A]
- **Point of Contact:** [Momchil Hardalov](mailto:hardalov@fmi.uni-sofia.bg)

### Dataset Summary

Recently, reading comprehension models achieved near-human performance on large-scale datasets such as SQuAD, CoQA, MS Macro, RACE, etc. This is largely due to the release of pre-trained contextualized representations such as BERT and ELMo, which can be fine-tuned for the target task. Despite those advances and the creation of more challenging datasets, most of the work is still done for English. Here, we study the effectiveness of multilingual BERT fine-tuned on large-scale English datasets for reading comprehension (e.g., for RACE), and we apply it to Bulgarian multiple-choice reading comprehension. We propose a new dataset containing 2,221 questions from matriculation exams for twelfth grade in various subjects -history, biology, geography and philosophy-, and 412 additional questions from online quizzes in history. While the quiz authors gave no relevant context, we incorporate knowledge from Wikipedia, retrieving documents matching the combination of question + each answer option.

### Supported Tasks and Leaderboards

[Needs More Information]

### Languages

Bulgarian

## Dataset Structure

### Data Instances

A typical data point comprises of  question sentence and 4 possible choice answers and the correct answer.

```
          {
            "id": "21181dda96414fd9b7a5e336ad84b45d",
            "qid": 1,
            "question": "!0<>AB>OB5;=> AJI5AB2C20I8 6828 A8AB5<8 A0:",
            "answers": [
              "28@CA8B5",
              "BJ:0=8B5",
              "<8B>E>=4@88B5",
              "54=>:;5BJG=8B5 >@30=87<8"
            ],
            "correct": "54=>:;5BJG=8B5 >@30=87<8",
            "url": "http://zamatura.eu/files/dzi/biologiq/2010/matura-biologiq-2010.pdf"
          },
```

### Data Fields

- url : A string having the url from which the question has been sourced from
- id: A string question identifier for each example
- qid: An integer which shows the sequence of the question in that particular URL
- question: The title of the question
- answers: A list of each answers
- correct: The correct answer

### Data Splits

The dataset covers the following domains

| Domain | #QA-paris | #Choices | Len Question | Len Options | Vocab Size |
|:-------|:---------:|:--------:|:------------:|:-----------:|:----------:|
| **12th Grade Matriculation Exam** |
| Biology | 437 | 4 | 10.44 | 2.64 | 2,414 (12,922)|
| Philosophy | 630 | 4 | 8.91 | 2.94| 3,636  (20,392) |
| Geography | 612 | 4 | 12.83 | 2.47 | 3,239 (17,668) |
| History | 542 | 4 | 23.74 | 3.64 | 5,466 (20,456) |
| **Online History Quizzes** |
| Bulgarian History | 229 | 4 | 14.05 | 2.80 | 2,287 (10,620) |
| PzHistory | 183 | 3 | 38.89 | 2.44 | 1,261 (7,518) |
| **Total** | 2,633 | 3.93 | 15.67 | 2.89 | 13,329 (56,104) |


## Dataset Creation

### Curation Rationale

The dataset has been curated from matriculation exams and online quizzes. These questions cover a large variety of science topics in biology, philosophy, geography, and history.

### Source Data

#### Initial Data Collection and Normalization

Data has been sourced from the matriculation exams and online quizzes.

#### Who are the source language producers?

[Needs More Information]

### Annotations

#### Annotation process

[Needs More Information]

#### Who are the annotators?

[Needs More Information]

### Personal and Sensitive Information

[Needs More Information]

## Considerations for Using the Data

### Social Impact of Dataset

[Needs More Information]

### Discussion of Biases

[Needs More Information]

### Other Known Limitations

[Needs More Information]

## Additional Information

### Dataset Curators

[Needs More Information]

### Licensing Information

[Needs More Information]

### Citation Information

```
@article{hardalov2019beyond,
  title={Beyond english-only reading comprehension: Experiments in zero-shot multilingual transfer for bulgarian},
  author={Hardalov, Momchil and Koychev, Ivan and Nakov, Preslav},
  journal={arXiv preprint arXiv:1908.01519},
  year={2019}
}
```

### Contributions

Thanks to [@saradhix](https://github.com/saradhix) for adding this dataset.
task_categories:question-answering task_ids:multiple-choice-qa annotations_creators:found language_creators:found multilinguality:monolingual size_categories:n<1K source_datasets:original language:bg license:apache-2.0 arxiv:1908.01519 region:us
{"annotations_creators": ["found"], "language_creators": ["found"], "language": ["bg"], "license": ["apache-2.0"], "multilinguality": ["monolingual"], "size_categories": ["n<1K"], "source_datasets": ["original"], "task_categories": ["question-answering"], "task_ids": ["multiple-choice-qa"], "pretty_name": "ReasoningBg", "dataset_info": [{"config_name": "biology-12th", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "qid", "dtype": "int32"}, {"name": "question", "dtype": "string"}, {"name": "answers", "sequence": "string"}, {"name": "correct", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 197725, "num_examples": 437}], "download_size": 1753795, "dataset_size": 197725}, {"config_name": "philosophy-12th", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "qid", "dtype": "int32"}, {"name": "question", "dtype": "string"}, {"name": "answers", "sequence": "string"}, {"name": "correct", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 286999, "num_examples": 630}], "download_size": 1753795, "dataset_size": 286999}, {"config_name": "geography-12th", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "qid", "dtype": "int32"}, {"name": "question", "dtype": "string"}, {"name": "answers", "sequence": "string"}, {"name": "correct", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 283417, "num_examples": 612}], "download_size": 1753795, "dataset_size": 283417}, {"config_name": "history-12th", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "qid", "dtype": "int32"}, {"name": "question", "dtype": "string"}, {"name": "answers", "sequence": "string"}, {"name": "correct", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 341472, "num_examples": 542}], "download_size": 1753795, "dataset_size": 341472}, {"config_name": "history-quiz", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "qid", "dtype": "int32"}, {"name": "question", "dtype": "string"}, {"name": "answers", "sequence": "string"}, {"name": "correct", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 164495, "num_examples": 412}], "download_size": 1753795, "dataset_size": 164495}]}
TAGS
#task_categories-question-answering #task_ids-multiple-choice-qa #annotations_creators-found #language_creators-found #multilinguality-monolingual #size_categories-n<1K #source_datasets-original #language-Bulgarian #license-apache-2.0 #arxiv-1908.01519 #region-us 

Dataset Card for reasoning\_bg
==============================


Table of Contents
-----------------


* Dataset Description
	+ Dataset Summary
	+ Supported Tasks and Leaderboards
	+ Languages
* Dataset Structure
	+ Data Instances
	+ Data Fields
	+ Data Splits
* Dataset Creation
	+ Curation Rationale
	+ Source Data
	+ Annotations
	+ Personal and Sensitive Information
* Considerations for Using the Data
	+ Social Impact of Dataset
	+ Discussion of Biases
	+ Other Known Limitations
* Additional Information
	+ Dataset Curators
	+ Licensing Information
	+ Citation Information
	+ Contributions


Dataset Description
-------------------


* Homepage: URL
* Repository: URL
* Paper: Beyond English-Only Reading Comprehension: Experiments in Zero-Shot Multilingual Transfer for Bulgarian
* Leaderboard: [N/A]
* Point of Contact: Momchil Hardalov


### Dataset Summary


Recently, reading comprehension models achieved near-human performance on large-scale datasets such as SQuAD, CoQA, MS Macro, RACE, etc. This is largely due to the release of pre-trained contextualized representations such as BERT and ELMo, which can be fine-tuned for the target task. Despite those advances and the creation of more challenging datasets, most of the work is still done for English. Here, we study the effectiveness of multilingual BERT fine-tuned on large-scale English datasets for reading comprehension (e.g., for RACE), and we apply it to Bulgarian multiple-choice reading comprehension. We propose a new dataset containing 2,221 questions from matriculation exams for twelfth grade in various subjects -history, biology, geography and philosophy-, and 412 additional questions from online quizzes in history. While the quiz authors gave no relevant context, we incorporate knowledge from Wikipedia, retrieving documents matching the combination of question + each answer option.


### Supported Tasks and Leaderboards


### Languages


Bulgarian


Dataset Structure
-----------------


### Data Instances


A typical data point comprises of question sentence and 4 possible choice answers and the correct answer.


### Data Fields


* url : A string having the url from which the question has been sourced from
* id: A string question identifier for each example
* qid: An integer which shows the sequence of the question in that particular URL
* question: The title of the question
* answers: A list of each answers
* correct: The correct answer


### Data Splits


The dataset covers the following domains



Dataset Creation
----------------


### Curation Rationale


The dataset has been curated from matriculation exams and online quizzes. These questions cover a large variety of science topics in biology, philosophy, geography, and history.


### Source Data


#### Initial Data Collection and Normalization


Data has been sourced from the matriculation exams and online quizzes.


#### Who are the source language producers?


### Annotations


#### Annotation process


#### Who are the annotators?


### Personal and Sensitive Information


Considerations for Using the Data
---------------------------------


### Social Impact of Dataset


### Discussion of Biases


### Other Known Limitations


Additional Information
----------------------


### Dataset Curators


### Licensing Information


### Contributions


Thanks to @saradhix for adding this dataset.



### Dataset Summary


Recently, reading comprehension models achieved near-human performance on large-scale datasets such as SQuAD, CoQA, MS Macro, RACE, etc. This is largely due to the release of pre-trained contextualized representations such as BERT and ELMo, which can be fine-tuned for the target task. Despite those advances and the creation of more challenging datasets, most of the work is still done for English. Here, we study the effectiveness of multilingual BERT fine-tuned on large-scale English datasets for reading comprehension (e.g., for RACE), and we apply it to Bulgarian multiple-choice reading comprehension. We propose a new dataset containing 2,221 questions from matriculation exams for twelfth grade in various subjects -history, biology, geography and philosophy-, and 412 additional questions from online quizzes in history. While the quiz authors gave no relevant context, we incorporate knowledge from Wikipedia, retrieving documents matching the combination of question + each answer option. ### Supported Tasks and Leaderboards ### Languages


Bulgarian


Dataset Structure
----------------- ### Data Instances


A typical data point comprises of question sentence and 4 possible choice answers and the correct answer. ### Data Fields


* url : A string having the url from which the question has been sourced from
* id: A string question identifier for each example
* qid: An integer which shows the sequence of the question in that particular URL
* question: The title of the question
* answers: A list of each answers
* correct: The correct answer ### Data Splits


The dataset covers the following domains



Dataset Creation
---------------- ### Curation Rationale


The dataset has been curated from matriculation exams and online quizzes. These questions cover a large variety of science topics in biology, philosophy, geography, and history. ### Source Data #### Initial Data Collection and Normalization


Data has been sourced from the matriculation exams and online quizzes. #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators ### Licensing Information ### Contributions


Thanks to @saradhix for adding this dataset.
TAGS
#task_categories-question-answering #task_ids-multiple-choice-qa #annotations_creators-found #language_creators-found #multilinguality-monolingual #size_categories-n<1K #source_datasets-original #language-Bulgarian #license-apache-2.0 #arxiv-1908.01519 #region-us 
 ### Dataset Summary


Recently, reading comprehension models achieved near-human performance on large-scale datasets such as SQuAD, CoQA, MS Macro, RACE, etc. This is largely due to the release of pre-trained contextualized representations such as BERT and ELMo, which can be fine-tuned for the target task. Despite those advances and the creation of more challenging datasets, most of the work is still done for English. Here, we study the effectiveness of multilingual BERT fine-tuned on large-scale English datasets for reading comprehension (e.g., for RACE), and we apply it to Bulgarian multiple-choice reading comprehension. We propose a new dataset containing 2,221 questions from matriculation exams for twelfth grade in various subjects -history, biology, geography and philosophy-, and 412 additional questions from online quizzes in history. While the quiz authors gave no relevant context, we incorporate knowledge from Wikipedia, retrieving documents matching the combination of question + each answer option. ### Supported Tasks and Leaderboards ### Languages


Bulgarian


Dataset Structure
----------------- ### Data Instances


A typical data point comprises of question sentence and 4 possible choice answers and the correct answer. ### Data Fields


* url : A string having the url from which the question has been sourced from
* id: A string question identifier for each example
* qid: An integer which shows the sequence of the question in that particular URL
* question: The title of the question
* answers: A list of each answers
* correct: The correct answer ### Data Splits


The dataset covers the following domains



Dataset Creation
---------------- ### Curation Rationale


The dataset has been curated from matriculation exams and online quizzes. These questions cover a large variety of science topics in biology, philosophy, geography, and history. ### Source Data #### Initial Data Collection and Normalization


Data has been sourced from the matriculation exams and online quizzes. #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators ### Licensing Information ### Contributions


Thanks to @saradhix for adding this dataset.
passage: TAGS
#task_categories-question-answering #task_ids-multiple-choice-qa #annotations_creators-found #language_creators-found #multilinguality-monolingual #size_categories-n<1K #source_datasets-original #language-Bulgarian #license-apache-2.0 #arxiv-1908.01519 #region-us 
### Dataset Summary


Recently, reading comprehension models achieved near-human performance on large-scale datasets such as SQuAD, CoQA, MS Macro, RACE, etc. This is largely due to the release of pre-trained contextualized representations such as BERT and ELMo, which can be fine-tuned for the target task. Despite those advances and the creation of more challenging datasets, most of the work is still done for English. Here, we study the effectiveness of multilingual BERT fine-tuned on large-scale English datasets for reading comprehension (e.g., for RACE), and we apply it to Bulgarian multiple-choice reading comprehension. We propose a new dataset containing 2,221 questions from matriculation exams for twelfth grade in various subjects -history, biology, geography and philosophy-, and 412 additional questions from online quizzes in history. While the quiz authors gave no relevant context, we incorporate knowledge from Wikipedia, retrieving documents matching the combination of question + each answer option.### Supported Tasks and Leaderboards### Languages


Bulgarian


Dataset Structure
-----------------### Data Instances


A typical data point comprises of question sentence and 4 possible choice answers and the correct answer.### Data Fields


* url : A string having the url from which the question has been sourced from
* id: A string question identifier for each example
* qid: An integer which shows the sequence of the question in that particular URL
* question: The title of the question
* answers: A list of each answers
* correct: The correct answer### Data Splits


The dataset covers the following domains



Dataset Creation
----------------