
# Dataset Card for Flores 101

## Table of Contents

- [Dataset Card for Flores 101](#dataset-card-for-flores-101)
  - [Table of Contents](#table-of-contents)
  - [Dataset Description](#dataset-description)
    - [Dataset Summary](#dataset-summary)
    - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
    - [Languages](#languages)
  - [Dataset Structure](#dataset-structure)
    - [Data Instances](#data-instances)
    - [Data Fields](#data-fields)
    - [Data Splits](#data-splits)
    - [Dataset Creation](#dataset-creation)
  - [Additional Information](#additional-information)
    - [Dataset Curators](#dataset-curators)
    - [Licensing Information](#licensing-information)
    - [Citation Information](#citation-information)

## Dataset Description

- **Home:** [WMT](http://www.statmt.org/wmt21/large-scale-multilingual-translation-task.html)
- **Repository:** [Github](https://github.com/facebookresearch/flores)
- **Blogpost:** [FAIR](https://ai.facebook.com/blog/the-flores-101-data-set-helping-build-better-translation-systems-around-the-world)
- **Paper:** [Arxiv](https://arxiv.org/abs/2106.03193)
- **Point of Contact:** [flores@fb.com](mailto:flores@fb.com)
- **Leaderboard** [Dynabench](https://dynabench.org/flores/Flores%20MT%20Evaluation%20(FULL))

### Dataset Summary

FLORES is a benchmark dataset for machine translation between English and low-resource languages.

Abstract from the original paper:

> One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the FLORES evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond.

**Disclaimer**: *The Flores-101 dataset is hosted by the Facebook and licensed under the [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/).

### Supported Tasks and Leaderboards

#### Multilingual Machine Translation

Refer to the [Dynabench leaderboard](https://dynabench.org/flores/Flores%20MT%20Evaluation%20(FULL)) for additional details on model evaluation on FLORES-101 in the context of the WMT2021 shared task on [Large-Scale Multilingual Machine Translation](http://www.statmt.org/wmt21/large-scale-multilingual-translation-task.html).

### Languages

The dataset contains parallel sentences for 101 languages, as mentioned in the original [Github](https://github.com/facebookresearch/flores/blob/master/README.md) page for the project. Languages are identified with the ISO 639-3 code (e.g. `eng`, `fra`, `rus`) as in the original dataset.

**New:** Use the configuration `all` to access the full set of parallel sentences for all the available languages in a single command.


## Dataset Structure

### Data Instances

A sample from the `dev` split for the Russian language (`rus` config) is provided below. All configurations have the same structure, and all sentences are aligned across configurations and splits.

```python
{
	'id': 1,
	'sentence': 'В понедельник ученые из Медицинской школы Стэнфордского университета объявили об изобретении нового диагностического инструмента, который может сортировать клетки по их типу; это маленький чип, который можно напечатать, используя стандартный струйный принтер примерно за 1 цент США.',
	'URL': 'https://en.wikinews.org/wiki/Scientists_say_new_medical_diagnostic_chip_can_sort_cells_anywhere_with_an_inkjet',
	'domain': 'wikinews',
	'topic': 'health',
	'has_image': 0,
	'has_hyperlink': 0
}
```

The text is provided as-in the original dataset, without further preprocessing or tokenization.

### Data Fields

- `id`: Row number for the data entry, starting at 1.
- `sentence`: The full sentence in the specific language.
- `URL`: The URL for the English article from which the sentence was extracted.
- `domain`: The domain of the sentence.
- `topic`: The topic of the sentence.
- `has_image`: Whether the  original article contains an image.
- `has_hyperlink`: Whether the  sentence contains a hyperlink.

### Data Splits

|            config| `dev`| `devtest`|
|-----------------:|-----:|---------:|
|all configurations|   997|     1012:|

### Dataset Creation

Please refer to the original article [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://arxiv.org/abs/2106.03193) for additional information on dataset creation.

## Additional Information

### Dataset Curators

The original authors of FLORES-101 are the curators of the original dataset. For problems or updates on this 🤗 Datasets version, please contact [gabriele.sarti996@gmail.com](mailto:gabriele.sarti996@gmail.com).

### Licensing Information

Licensed with Creative Commons Attribution Share Alike 4.0. License available [here](https://creativecommons.org/licenses/by-sa/4.0/).

### Citation Information

Please cite the authors if you use these corpora in your work:

```bibtex
@inproceedings{flores101,
  title={The FLORES-101  Evaluation Benchmark for Low-Resource and Multilingual Machine Translation},
  author={Goyal, Naman and Gao, Cynthia and Chaudhary, Vishrav and Chen, Peng-Jen and Wenzek, Guillaume and Ju, Da and Krishnan, Sanjana and Ranzato, Marc'Aurelio and Guzm\'{a}n, Francisco and Fan, Angela},
  journal={arXiv preprint arXiv:2106.03193},
  year={2021}
}
```
task_categories:text-generation task_categories:translation annotations_creators:found language_creators:expert-generated multilinguality:multilingual multilinguality:translation size_categories:unknown source_datasets:extended|flores language:af language:am language:ar language:hy language:as language:ast language:az language:be language:bn language:bs language:bg language:my language:ca language:ceb language:zho language:hr language:cs language:da language:nl language:en language:et language:tl language:fi language:fr language:ff language:gl language:lg language:ka language:de language:el language:gu language:ha language:he language:hi language:hu language:is language:ig language:id language:ga language:it language:ja language:jv language:kea language:kam language:kn language:kk language:km language:ko language:ky language:lo language:lv language:ln language:lt language:luo language:lb language:mk language:ms language:ml language:mt language:mi language:mr language:mn language:ne language:ns language:no language:ny language:oc language:or language:om language:ps language:fa language:pl language:pt language:pa language:ro language:ru language:sr language:sn language:sd language:sk language:sl language:so language:ku language:es language:sw language:sv language:tg language:ta language:te language:th language:tr language:uk language:umb language:ur language:uz language:vi language:cy language:wo language:xh language:yo language:zu license:cc-by-sa-4.0 conditional-text-generation arxiv:2106.03193 region:us
{"annotations_creators": ["found"], "language_creators": ["expert-generated"], "language": ["af", "am", "ar", "hy", "as", "ast", "az", "be", "bn", "bs", "bg", "my", "ca", "ceb", "zho", "hr", "cs", "da", "nl", "en", "et", "tl", "fi", "fr", "ff", "gl", "lg", "ka", "de", "el", "gu", "ha", "he", "hi", "hu", "is", "ig", "id", "ga", "it", "ja", "jv", "kea", "kam", "kn", "kk", "km", "ko", "ky", "lo", "lv", "ln", "lt", "luo", "lb", "mk", "ms", "ml", "mt", "mi", "mr", "mn", "ne", "ns", "no", "ny", "oc", "or", "om", "ps", "fa", "pl", "pt", "pa", "ro", "ru", "sr", "sn", "sd", "sk", "sl", "so", "ku", "es", "sw", "sv", "tg", "ta", "te", "th", "tr", "uk", "umb", "ur", "uz", "vi", "cy", "wo", "xh", "yo", "zu"], "license": ["cc-by-sa-4.0"], "multilinguality": ["multilingual", "translation"], "size_categories": ["unknown"], "source_datasets": ["extended|flores"], "task_categories": ["text-generation", "translation"], "task_ids": [], "paperswithcode_id": "flores", "pretty_name": "flores101", "tags": ["conditional-text-generation"]}
TAGS
#task_categories-text-generation #task_categories-translation #annotations_creators-found #language_creators-expert-generated #multilinguality-multilingual #multilinguality-translation #size_categories-unknown #source_datasets-extended|flores #language-Afrikaans #language-Amharic #language-Arabic #language-Armenian #language-Assamese #language-Asturian #language-Azerbaijani #language-Belarusian #language-Bengali #language-Bosnian #language-Bulgarian #language-Burmese #language-Catalan #language-Cebuano #language-Chinese #language-Croatian #language-Czech #language-Danish #language-Dutch #language-English #language-Estonian #language-Tagalog #language-Finnish #language-French #language-Fulah #language-Galician #language-Ganda #language-Georgian #language-German #language-Modern Greek (1453-) #language-Gujarati #language-Hausa #language-Hebrew #language-Hindi #language-Hungarian #language-Icelandic #language-Igbo #language-Indonesian #language-Irish #language-Italian #language-Japanese #language-Javanese #language-Kabuverdianu #language-Kamba (Kenya) #language-Kannada #language-Kazakh #language-Khmer #language-Korean #language-Kirghiz #language-Lao #language-Latvian #language-Lingala #language-Lithuanian #language-Luo (Kenya and Tanzania) #language-Luxembourgish #language-Macedonian #language-Malay (macrolanguage) #language-Malayalam #language-Maltese #language-Maori #language-Marathi #language-Mongolian #language-Nepali (macrolanguage) #language-ns #language-Norwegian #language-Nyanja #language-Occitan (post 1500) #language-Oriya (macrolanguage) #language-Oromo #language-Pushto #language-Persian #language-Polish #language-Portuguese #language-Panjabi #language-Romanian #language-Russian #language-Serbian #language-Shona #language-Sindhi #language-Slovak #language-Slovenian #language-Somali #language-Kurdish #language-Spanish #language-Swahili (macrolanguage) #language-Swedish #language-Tajik #language-Tamil #language-Telugu #language-Thai #language-Turkish #language-Ukrainian #language-Umbundu #language-Urdu #language-Uzbek #language-Vietnamese #language-Welsh #language-Wolof #language-Xhosa #language-Yoruba #language-Zulu #license-cc-by-sa-4.0 #conditional-text-generation #arxiv-2106.03193 #region-us 

Dataset Card for Flores 101
===========================


Table of Contents
-----------------


* Dataset Card for Flores 101
	+ Table of Contents
	+ Dataset Description
		- Dataset Summary
		- Supported Tasks and Leaderboards
		- Languages
	+ Dataset Structure
		- Data Instances
		- Data Fields
		- Data Splits
		- Dataset Creation
	+ Additional Information
		- Dataset Curators
		- Licensing Information
		- Citation Information


Dataset Description
-------------------


* Home: WMT
* Repository: Github
* Blogpost: FAIR
* Paper: Arxiv
* Point of Contact: flores@URL
* Leaderboard Dynabench)


### Dataset Summary


FLORES is a benchmark dataset for machine translation between English and low-resource languages.


Abstract from the original paper:



> 
> One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the FLORES evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond.
> 
> 
> 


Disclaimer: \*The Flores-101 dataset is hosted by the Facebook and licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.


### Supported Tasks and Leaderboards


#### Multilingual Machine Translation


Refer to the Dynabench leaderboard) for additional details on model evaluation on FLORES-101 in the context of the WMT2021 shared task on Large-Scale Multilingual Machine Translation.


### Languages


The dataset contains parallel sentences for 101 languages, as mentioned in the original Github page for the project. Languages are identified with the ISO 639-3 code (e.g. 'eng', 'fra', 'rus') as in the original dataset.


New: Use the configuration 'all' to access the full set of parallel sentences for all the available languages in a single command.


Dataset Structure
-----------------


### Data Instances


A sample from the 'dev' split for the Russian language ('rus' config) is provided below. All configurations have the same structure, and all sentences are aligned across configurations and splits.


The text is provided as-in the original dataset, without further preprocessing or tokenization.


### Data Fields


* 'id': Row number for the data entry, starting at 1.
* 'sentence': The full sentence in the specific language.
* 'URL': The URL for the English article from which the sentence was extracted.
* 'domain': The domain of the sentence.
* 'topic': The topic of the sentence.
* 'has\_image': Whether the original article contains an image.
* 'has\_hyperlink': Whether the sentence contains a hyperlink.


### Data Splits



### Dataset Creation


Please refer to the original article The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation for additional information on dataset creation.


Additional Information
----------------------


### Dataset Curators


The original authors of FLORES-101 are the curators of the original dataset. For problems or updates on this Datasets version, please contact gabriele.sarti996@URL.


### Licensing Information


Licensed with Creative Commons Attribution Share Alike 4.0. License available here.


Please cite the authors if you use these corpora in your work:



### Dataset Summary


FLORES is a benchmark dataset for machine translation between English and low-resource languages.


Abstract from the original paper:



> 
> One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the FLORES evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond.
> 
> 
> 


Disclaimer: \*The Flores-101 dataset is hosted by the Facebook and licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. ### Supported Tasks and Leaderboards #### Multilingual Machine Translation


Refer to the Dynabench leaderboard) for additional details on model evaluation on FLORES-101 in the context of the WMT2021 shared task on Large-Scale Multilingual Machine Translation. ### Languages


The dataset contains parallel sentences for 101 languages, as mentioned in the original Github page for the project. Languages are identified with the ISO 639-3 code (e.g. 'eng', 'fra', 'rus') as in the original dataset.


New: Use the configuration 'all' to access the full set of parallel sentences for all the available languages in a single command.


Dataset Structure
----------------- ### Data Instances


A sample from the 'dev' split for the Russian language ('rus' config) is provided below. All configurations have the same structure, and all sentences are aligned across configurations and splits.


The text is provided as-in the original dataset, without further preprocessing or tokenization. ### Data Fields


* 'id': Row number for the data entry, starting at 1.
* 'sentence': The full sentence in the specific language.
* 'URL': The URL for the English article from which the sentence was extracted.
* 'domain': The domain of the sentence.
* 'topic': The topic of the sentence.
* 'has\_image': Whether the original article contains an image.
* 'has\_hyperlink': Whether the sentence contains a hyperlink. ### Data Splits ### Dataset Creation


Please refer to the original article The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation for additional information on dataset creation.


Additional Information
---------------------- ### Dataset Curators


The original authors of FLORES-101 are the curators of the original dataset. For problems or updates on this Datasets version, please contact gabriele.sarti996@URL. ### Licensing Information


Licensed with Creative Commons Attribution Share Alike 4.0. License available here.


Please cite the authors if you use these corpora in your work:
TAGS
#task_categories-text-generation #task_categories-translation #annotations_creators-found #language_creators-expert-generated #multilinguality-multilingual #multilinguality-translation #size_categories-unknown #source_datasets-extended|flores #language-Afrikaans #language-Amharic #language-Arabic #language-Armenian #language-Assamese #language-Asturian #language-Azerbaijani #language-Belarusian #language-Bengali #language-Bosnian #language-Bulgarian #language-Burmese #language-Catalan #language-Cebuano #language-Chinese #language-Croatian #language-Czech #language-Danish #language-Dutch #language-English #language-Estonian #language-Tagalog #language-Finnish #language-French #language-Fulah #language-Galician #language-Ganda #language-Georgian #language-German #language-Modern Greek (1453-) #language-Gujarati #language-Hausa #language-Hebrew #language-Hindi #language-Hungarian #language-Icelandic #language-Igbo #language-Indonesian #language-Irish #language-Italian #language-Japanese #language-Javanese #language-Kabuverdianu #language-Kamba (Kenya) #language-Kannada #language-Kazakh #language-Khmer #language-Korean #language-Kirghiz #language-Lao #language-Latvian #language-Lingala #language-Lithuanian #language-Luo (Kenya and Tanzania) #language-Luxembourgish #language-Macedonian #language-Malay (macrolanguage) #language-Malayalam #language-Maltese #language-Maori #language-Marathi #language-Mongolian #language-Nepali (macrolanguage) #language-ns #language-Norwegian #language-Nyanja #language-Occitan (post 1500) #language-Oriya (macrolanguage) #language-Oromo #language-Pushto #language-Persian #language-Polish #language-Portuguese #language-Panjabi #language-Romanian #language-Russian #language-Serbian #language-Shona #language-Sindhi #language-Slovak #language-Slovenian #language-Somali #language-Kurdish #language-Spanish #language-Swahili (macrolanguage) #language-Swedish #language-Tajik #language-Tamil #language-Telugu #language-Thai #language-Turkish #language-Ukrainian #language-Umbundu #language-Urdu #language-Uzbek #language-Vietnamese #language-Welsh #language-Wolof #language-Xhosa #language-Yoruba #language-Zulu #license-cc-by-sa-4.0 #conditional-text-generation #arxiv-2106.03193 #region-us 
 ### Dataset Summary


FLORES is a benchmark dataset for machine translation between English and low-resource languages.


Abstract from the original paper:



> 
> One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the FLORES evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond.
> 
> 
> 


Disclaimer: \*The Flores-101 dataset is hosted by the Facebook and licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. ### Supported Tasks and Leaderboards #### Multilingual Machine Translation


Refer to the Dynabench leaderboard) for additional details on model evaluation on FLORES-101 in the context of the WMT2021 shared task on Large-Scale Multilingual Machine Translation. ### Languages


The dataset contains parallel sentences for 101 languages, as mentioned in the original Github page for the project. Languages are identified with the ISO 639-3 code (e.g. 'eng', 'fra', 'rus') as in the original dataset.


New: Use the configuration 'all' to access the full set of parallel sentences for all the available languages in a single command.


Dataset Structure
----------------- ### Data Instances


A sample from the 'dev' split for the Russian language ('rus' config) is provided below. All configurations have the same structure, and all sentences are aligned across configurations and splits.


The text is provided as-in the original dataset, without further preprocessing or tokenization. ### Data Fields


* 'id': Row number for the data entry, starting at 1.
* 'sentence': The full sentence in the specific language.
* 'URL': The URL for the English article from which the sentence was extracted.
* 'domain': The domain of the sentence.
* 'topic': The topic of the sentence.
* 'has\_image': Whether the original article contains an image.
* 'has\_hyperlink': Whether the sentence contains a hyperlink. ### Data Splits ### Dataset Creation


Please refer to the original article The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation for additional information on dataset creation.


Additional Information
---------------------- ### Dataset Curators


The original authors of FLORES-101 are the curators of the original dataset. For problems or updates on this Datasets version, please contact gabriele.sarti996@URL. ### Licensing Information


Licensed with Creative Commons Attribution Share Alike 4.0. License available here.


Please cite the authors if you use these corpora in your work: