

# Xtr-WikiQA

## Table of Contents
- [Dataset Card Creation Guide](#dataset-card-creation-guide)
  - [Table of Contents](#table-of-contents)
  - [Dataset Description](#dataset-description)
    - [Dataset Summary](#dataset-summary)
    - [Languages](#languages)
  - [Dataset Structure](#dataset-structure)
    - [Data Instances](#data-instances)
    - [Data Fields](#data-fields)
    - [Data Splits](#data-splits)
  - [Dataset Creation](#dataset-creation)
    - [Source Data](#source-data)
  - [Additional Information](#additional-information)
    - [Licensing Information](#licensing-information)
    - [Citation Information](#citation-information)
    - [Contributions](#contributions)

## Dataset Description

- **Homepage:** [Amazon Science](https://www.amazon.science/publications/cross-lingual-knowledge-distillation-for-answer-sentence-selection-in-low-resource-languages)
- **Paper:** [Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages](https://aclanthology.org/2023.findings-acl.885/)
- **Point of Contact:** [Yoshitomo Matsubara](yomtsub@amazon.com)

### Dataset Summary

***Xtr-WikiQA*** is an Answer Sentence Selection (AS2) dataset in 9 non-English languages, proposed in our paper accepted at ACL 2023 (Findings): [**Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages**](https://aclanthology.org/2023.findings-acl.885/).
This dataset is based on an English AS2 dataset, WikiQA ([Original](https://msropendata.com/datasets/21032bb1-88bd-4656-9570-3172ae1757f0), [Hugging Face](https://huggingface.co/datasets/wiki_qa)).
For translations, we used [Amazon Translate](https://aws.amazon.com/translate/).

### Languages

- Arabic (ar)
- Spanish (es)
- French (fr)
- German (de)
- Hindi (hi)
- Italian (it)
- Japanese (ja)
- Dutch (nl)
- Portuguese (pt)

File location: [`tsv/`](https://huggingface.co/datasets/AmazonScience/xtr-wiki_qa/tree/main/tsv)

## Dataset Structure

### Data Instances

This is an example instance from the Arabic training split of Xtr-WikiQA dataset. 

```
{
  "QuestionID": "Q1",
  "Question": "كيف تتشكل الكهوف الجليدية؟",
  "DocumentID": "D1",
  "DocumentTitle": "كهف جليدي",
  "SentenceID": "D1-0",
  "Sentence": "كهف جليدي مغمور جزئيًا على نهر بيريتو مورينو الجليدي.",
  "Label": 0
}
```

All the translated instances in tsv files are listed in the same order of the original (native) instances in the WikiQA dataset.

For example, the 2nd instance in [`tsv/ar-train.tsv`](https://huggingface.co/datasets/AmazonScience/xtr-wiki_qa/blob/main/tsv/ar-train.tsv) (Arabic-translated from English)
corresponds to the 2nd instance in [`WikiQA-train.tsv`](https://msropendata.com/datasets/21032bb1-88bd-4656-9570-3172ae1757f0) (English).

### Data Fields

Each instance (a QA pair) consists of the following fields:

- `QuestionID`: Question ID (str)
- `Question`: Question to be answered (str)
- `DocumentID`: Document ID (str)
- `DocumentTitle`: Document title (str)
- `SentenceID`: Answer sentence in the document (str)
- `Sentence`: Answer sentence in the document (str)
- `Label`: Label that indicates the answer sentence correctly answers the question (int, 1: correct, 0: incorrect)


### Data Splits

|                   |             | **#Questions** |          |   |           | **#Sentences** |          |
|-------------------|------------:|---------------:|---------:|---|----------:|---------------:|---------:|
|                   |   **train** |        **dev** | **test** |   | **train** |        **dev** | **test** |
| **Each language** |         873 |            126 |      243 |   |     8,671 |          1,130 |    2,351 |

See [our paper](#citation-information) for more details about the statistics of the datasets.


## Dataset Creation

### Source Data

The source of Xtr-WikiQA dataset is [WikiQA](https://msropendata.com/datasets/21032bb1-88bd-4656-9570-3172ae1757f0).


## Additional Information

### Licensing Information

[CDLA-Permissive-2.0](LICENSE.md)

### Citation Information

```bibtex
@inproceedings{gupta2023cross-lingual,
  title={{Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages}},
  author={Gupta, Shivanshu and Matsubara, Yoshitomo and Chadha, Ankit and Moschitti, Alessandro},
  booktitle={Findings of the Association for Computational Linguistics: ACL 2023},
  pages={14078--14092},
  year={2023}
}
```


### Contributions

- [Shivanshu Gupta](https://huggingface.co/shivanshu)
- [Yoshitomo Matsubara](https://huggingface.co/yoshitomo-matsubara)
- Ankit Chadha
- Alessandro Moschitti
task_categories:question-answering task_categories:text-retrieval task_ids:open-domain-qa annotations_creators:machine-generated language_creators:found multilinguality:multilingual multilinguality:translation size_categories:100K<n<1M source_datasets:extended|wiki_qa language:ar language:es language:fr language:de language:hi language:it language:ja language:nl language:pt license:cdla-permissive-2.0 as2 answer sentence selection text retrieval question answering region:us
{"annotations_creators": ["machine-generated"], "language_creators": ["found"], "language": ["ar", "es", "fr", "de", "hi", "it", "ja", "nl", "pt"], "license": "cdla-permissive-2.0", "multilinguality": ["multilingual", "translation"], "size_categories": ["100K<n<1M"], "source_datasets": ["extended|wiki_qa"], "task_categories": ["question-answering", "text-retrieval"], "task_ids": ["open-domain-qa"], "pretty_name": "xtr-wiki_qa", "license_details": "https://huggingface.co/datasets/AmazonScience/xtr-wiki_qa/blob/main/LICENSE.md", "tags": ["as2", "answer sentence selection", "text retrieval", "question answering"]}
TAGS
#task_categories-question-answering #task_categories-text-retrieval #task_ids-open-domain-qa #annotations_creators-machine-generated #language_creators-found #multilinguality-multilingual #multilinguality-translation #size_categories-100K<n<1M #source_datasets-extended|wiki_qa #language-Arabic #language-Spanish #language-French #language-German #language-Hindi #language-Italian #language-Japanese #language-Dutch #language-Portuguese #license-cdla-permissive-2.0 #as2 #answer sentence selection #text retrieval #question answering #region-us 

Xtr-WikiQA
==========


Table of Contents
-----------------


* Dataset Card Creation Guide
	+ Table of Contents
	+ Dataset Description
		- Dataset Summary
		- Languages
	+ Dataset Structure
		- Data Instances
		- Data Fields
		- Data Splits
	+ Dataset Creation
		- Source Data
	+ Additional Information
		- Licensing Information
		- Citation Information
		- Contributions


Dataset Description
-------------------


* Homepage: Amazon Science
* Paper: Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages
* Point of Contact: Yoshitomo Matsubara


### Dataset Summary


*Xtr-WikiQA* is an Answer Sentence Selection (AS2) dataset in 9 non-English languages, proposed in our paper accepted at ACL 2023 (Findings): Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages.
This dataset is based on an English AS2 dataset, WikiQA (Original, Hugging Face).
For translations, we used Amazon Translate.


### Languages


* Arabic (ar)
* Spanish (es)
* French (fr)
* German (de)
* Hindi (hi)
* Italian (it)
* Japanese (ja)
* Dutch (nl)
* Portuguese (pt)


File location: 'tsv/'


Dataset Structure
-----------------


### Data Instances


This is an example instance from the Arabic training split of Xtr-WikiQA dataset.


All the translated instances in tsv files are listed in the same order of the original (native) instances in the WikiQA dataset.


For example, the 2nd instance in 'tsv/URL' (Arabic-translated from English)
corresponds to the 2nd instance in 'URL' (English).


### Data Fields


Each instance (a QA pair) consists of the following fields:


* 'QuestionID': Question ID (str)
* 'Question': Question to be answered (str)
* 'DocumentID': Document ID (str)
* 'DocumentTitle': Document title (str)
* 'SentenceID': Answer sentence in the document (str)
* 'Sentence': Answer sentence in the document (str)
* 'Label': Label that indicates the answer sentence correctly answers the question (int, 1: correct, 0: incorrect)


### Data Splits



See our paper for more details about the statistics of the datasets.


Dataset Creation
----------------


### Source Data


The source of Xtr-WikiQA dataset is WikiQA.


Additional Information
----------------------


### Licensing Information


CDLA-Permissive-2.0


### Contributions


* Shivanshu Gupta
* Yoshitomo Matsubara
* Ankit Chadha
* Alessandro Moschitti



### Dataset Summary


*Xtr-WikiQA* is an Answer Sentence Selection (AS2) dataset in 9 non-English languages, proposed in our paper accepted at ACL 2023 (Findings): Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages.
This dataset is based on an English AS2 dataset, WikiQA (Original, Hugging Face).
For translations, we used Amazon Translate. ### Languages


* Arabic (ar)
* Spanish (es)
* French (fr)
* German (de)
* Hindi (hi)
* Italian (it)
* Japanese (ja)
* Dutch (nl)
* Portuguese (pt)


File location: 'tsv/'


Dataset Structure
----------------- ### Data Instances


This is an example instance from the Arabic training split of Xtr-WikiQA dataset.


All the translated instances in tsv files are listed in the same order of the original (native) instances in the WikiQA dataset.


For example, the 2nd instance in 'tsv/URL' (Arabic-translated from English)
corresponds to the 2nd instance in 'URL' (English). ### Data Fields


Each instance (a QA pair) consists of the following fields:


* 'QuestionID': Question ID (str)
* 'Question': Question to be answered (str)
* 'DocumentID': Document ID (str)
* 'DocumentTitle': Document title (str)
* 'SentenceID': Answer sentence in the document (str)
* 'Sentence': Answer sentence in the document (str)
* 'Label': Label that indicates the answer sentence correctly answers the question (int, 1: correct, 0: incorrect) ### Data Splits



See our paper for more details about the statistics of the datasets.


Dataset Creation
---------------- ### Source Data


The source of Xtr-WikiQA dataset is WikiQA.


Additional Information
---------------------- ### Licensing Information


CDLA-Permissive-2.0 ### Contributions


* Shivanshu Gupta
* Yoshitomo Matsubara
* Ankit Chadha
* Alessandro Moschitti
TAGS
#task_categories-question-answering #task_categories-text-retrieval #task_ids-open-domain-qa #annotations_creators-machine-generated #language_creators-found #multilinguality-multilingual #multilinguality-translation #size_categories-100K<n<1M #source_datasets-extended|wiki_qa #language-Arabic #language-Spanish #language-French #language-German #language-Hindi #language-Italian #language-Japanese #language-Dutch #language-Portuguese #license-cdla-permissive-2.0 #as2 #answer sentence selection #text retrieval #question answering #region-us 
 ### Dataset Summary


*Xtr-WikiQA* is an Answer Sentence Selection (AS2) dataset in 9 non-English languages, proposed in our paper accepted at ACL 2023 (Findings): Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages.
This dataset is based on an English AS2 dataset, WikiQA (Original, Hugging Face).
For translations, we used Amazon Translate. ### Languages


* Arabic (ar)
* Spanish (es)
* French (fr)
* German (de)
* Hindi (hi)
* Italian (it)
* Japanese (ja)
* Dutch (nl)
* Portuguese (pt)


File location: 'tsv/'


Dataset Structure
----------------- ### Data Instances


This is an example instance from the Arabic training split of Xtr-WikiQA dataset.


All the translated instances in tsv files are listed in the same order of the original (native) instances in the WikiQA dataset.


For example, the 2nd instance in 'tsv/URL' (Arabic-translated from English)
corresponds to the 2nd instance in 'URL' (English). ### Data Fields


Each instance (a QA pair) consists of the following fields:


* 'QuestionID': Question ID (str)
* 'Question': Question to be answered (str)
* 'DocumentID': Document ID (str)
* 'DocumentTitle': Document title (str)
* 'SentenceID': Answer sentence in the document (str)
* 'Sentence': Answer sentence in the document (str)
* 'Label': Label that indicates the answer sentence correctly answers the question (int, 1: correct, 0: incorrect) ### Data Splits



See our paper for more details about the statistics of the datasets.


Dataset Creation
---------------- ### Source Data


The source of Xtr-WikiQA dataset is WikiQA.


Additional Information
---------------------- ### Licensing Information


CDLA-Permissive-2.0 ### Contributions


* Shivanshu Gupta
* Yoshitomo Matsubara
* Ankit Chadha
* Alessandro Moschitti
passage: TAGS
#task_categories-question-answering #task_categories-text-retrieval #task_ids-open-domain-qa #annotations_creators-machine-generated #language_creators-found #multilinguality-multilingual #multilinguality-translation #size_categories-100K<n<1M #source_datasets-extended|wiki_qa #language-Arabic #language-Spanish #language-French #language-German #language-Hindi #language-Italian #language-Japanese #language-Dutch #language-Portuguese #license-cdla-permissive-2.0 #as2 #answer sentence selection #text retrieval #question answering #region-us 
### Dataset Summary


*Xtr-WikiQA* is an Answer Sentence Selection (AS2) dataset in 9 non-English languages, proposed in our paper accepted at ACL 2023 (Findings): Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages.
This dataset is based on an English AS2 dataset, WikiQA (Original, Hugging Face).
For translations, we used Amazon Translate.### Languages


* Arabic (ar)
* Spanish (es)
* French (fr)
* German (de)
* Hindi (hi)
* Italian (it)
* Japanese (ja)
* Dutch (nl)
* Portuguese (pt)


File location: 'tsv/'


Dataset Structure
-----------------### Data Instances


This is an example instance from the Arabic training split of Xtr-WikiQA dataset.


All the translated instances in tsv files are listed in the same order of the original (native) instances in the WikiQA dataset.


For example, the 2nd instance in 'tsv/URL' (Arabic-translated from English)
corresponds to the 2nd instance in 'URL' (English).