
# Dataset Summary

This dataset contains all DA human annotations from previous WMT News Translation shared tasks.

The data is organised into 8 columns:
- lp: language pair
- src: input text
- mt: translation
- ref: reference translation
- score: z score
- raw: direct assessment
- annotators: number of annotators
- domain: domain of the input text (e.g. news)
- year: collection year
  
You can also find the original data for each year in the results section https://www.statmt.org/wmt{YEAR}/results.html e.g: for 2020 data: [https://www.statmt.org/wmt20/results.html](https://www.statmt.org/wmt20/results.html)

## Python usage:

```python
from datasets import load_dataset
dataset = load_dataset("RicardoRei/wmt-da-human-evaluation", split="train")
```

There is no standard train/test split for this dataset but you can easily split it according to year, language pair or domain. E.g. :

```python
# split by year
data = dataset.filter(lambda example: example["year"] == 2022)

# split by LP
data = dataset.filter(lambda example: example["lp"] == "en-de")

# split by domain
data = dataset.filter(lambda example: example["domain"] == "news")
```

Note that most data is from News domain.


## Citation Information

If you use this data please cite the WMT findings from previous years:
- [Findings of the 2017 Conference on Machine Translation (WMT17)](https://aclanthology.org/W17-4717.pdf)
- [Findings of the 2018 Conference on Machine Translation (WMT18)](https://aclanthology.org/W18-6401.pdf)
- [Findings of the 2019 Conference on Machine Translation (WMT19)](https://aclanthology.org/W19-5301.pdf)
- [Findings of the 2020 Conference on Machine Translation (WMT20)](https://aclanthology.org/2020.wmt-1.1.pdf)
- [Findings of the 2021 Conference on Machine Translation (WMT21)](https://aclanthology.org/2021.wmt-1.1.pdf)
- [Findings of the 2022 Conference on Machine Translation (WMT22)](https://aclanthology.org/2022.wmt-1.1.pdf)
size_categories:1M<n<10M language:bn language:cs language:de language:en language:et language:fi language:fr language:gu language:ha language:hi language:is language:ja language:kk language:km language:lt language:lv language:pl language:ps language:ru language:ta language:tr language:uk language:xh language:zh language:zu license:apache-2.0 mt-evaluation WMT 41-lang-pairs region:us
{"language": ["bn", "cs", "de", "en", "et", "fi", "fr", "gu", "ha", "hi", "is", "ja", "kk", "km", "lt", "lv", "pl", "ps", "ru", "ta", "tr", "uk", "xh", "zh", "zu"], "license": "apache-2.0", "size_categories": ["1M<n<10M"], "tags": ["mt-evaluation", "WMT", "41-lang-pairs"]}
TAGS
#size_categories-1M<n<10M #language-Bengali #language-Czech #language-German #language-English #language-Estonian #language-Finnish #language-French #language-Gujarati #language-Hausa #language-Hindi #language-Icelandic #language-Japanese #language-Kazakh #language-Khmer #language-Lithuanian #language-Latvian #language-Polish #language-Pushto #language-Russian #language-Tamil #language-Turkish #language-Ukrainian #language-Xhosa #language-Chinese #language-Zulu #license-apache-2.0 #mt-evaluation #WMT #41-lang-pairs #region-us 


# Dataset Summary

This dataset contains all DA human annotations from previous WMT News Translation shared tasks.

The data is organised into 8 columns:
- lp: language pair
- src: input text
- mt: translation
- ref: reference translation
- score: z score
- raw: direct assessment
- annotators: number of annotators
- domain: domain of the input text (e.g. news)
- year: collection year
  
You can also find the original data for each year in the results section URL e.g: for 2020 data: URL

## Python usage:



There is no standard train/test split for this dataset but you can easily split it according to year, language pair or domain. E.g. :



Note that most data is from News domain.




If you use this data please cite the WMT findings from previous years:
- Findings of the 2017 Conference on Machine Translation (WMT17)
- Findings of the 2018 Conference on Machine Translation (WMT18)
- Findings of the 2019 Conference on Machine Translation (WMT19)
- Findings of the 2020 Conference on Machine Translation (WMT20)
- Findings of the 2021 Conference on Machine Translation (WMT21)
- Findings of the 2022 Conference on Machine Translation (WMT22)
# Dataset Summary

This dataset contains all DA human annotations from previous WMT News Translation shared tasks.

The data is organised into 8 columns:
- lp: language pair
- src: input text
- mt: translation
- ref: reference translation
- score: z score
- raw: direct assessment
- annotators: number of annotators
- domain: domain of the input text (e.g. news)
- year: collection year
  
You can also find the original data for each year in the results section URL e.g: for 2020 data: URL ## Python usage:



There is no standard train/test split for this dataset but you can easily split it according to year, language pair or domain. E.g. :



Note that most data is from News domain.




If you use this data please cite the WMT findings from previous years:
- Findings of the 2017 Conference on Machine Translation (WMT17)
- Findings of the 2018 Conference on Machine Translation (WMT18)
- Findings of the 2019 Conference on Machine Translation (WMT19)
- Findings of the 2020 Conference on Machine Translation (WMT20)
- Findings of the 2021 Conference on Machine Translation (WMT21)
- Findings of the 2022 Conference on Machine Translation (WMT22)
TAGS
#size_categories-1M<n<10M #language-Bengali #language-Czech #language-German #language-English #language-Estonian #language-Finnish #language-French #language-Gujarati #language-Hausa #language-Hindi #language-Icelandic #language-Japanese #language-Kazakh #language-Khmer #language-Lithuanian #language-Latvian #language-Polish #language-Pushto #language-Russian #language-Tamil #language-Turkish #language-Ukrainian #language-Xhosa #language-Chinese #language-Zulu #license-apache-2.0 #mt-evaluation #WMT #41-lang-pairs #region-us 
 # Dataset Summary

This dataset contains all DA human annotations from previous WMT News Translation shared tasks.

The data is organised into 8 columns:
- lp: language pair
- src: input text
- mt: translation
- ref: reference translation
- score: z score
- raw: direct assessment
- annotators: number of annotators
- domain: domain of the input text (e.g. news)
- year: collection year
  
You can also find the original data for each year in the results section URL e.g: for 2020 data: URL ## Python usage:



There is no standard train/test split for this dataset but you can easily split it according to year, language pair or domain. E.g. :



Note that most data is from News domain.




If you use this data please cite the WMT findings from previous years:
- Findings of the 2017 Conference on Machine Translation (WMT17)
- Findings of the 2018 Conference on Machine Translation (WMT18)
- Findings of the 2019 Conference on Machine Translation (WMT19)
- Findings of the 2020 Conference on Machine Translation (WMT20)
- Findings of the 2021 Conference on Machine Translation (WMT21)
- Findings of the 2022 Conference on Machine Translation (WMT22)
passage: TAGS
#size_categories-1M<n<10M #language-Bengali #language-Czech #language-German #language-English #language-Estonian #language-Finnish #language-French #language-Gujarati #language-Hausa #language-Hindi #language-Icelandic #language-Japanese #language-Kazakh #language-Khmer #language-Lithuanian #language-Latvian #language-Polish #language-Pushto #language-Russian #language-Tamil #language-Turkish #language-Ukrainian #language-Xhosa #language-Chinese #language-Zulu #license-apache-2.0 #mt-evaluation #WMT #41-lang-pairs #region-us 
# Dataset Summary

This dataset contains all DA human annotations from previous WMT News Translation shared tasks.

The data is organised into 8 columns:
- lp: language pair
- src: input text
- mt: translation
- ref: reference translation
- score: z score
- raw: direct assessment
- annotators: number of annotators
- domain: domain of the input text (e.g. news)
- year: collection year
  
You can also find the original data for each year in the results section URL e.g: for 2020 data: URL## Python usage:



There is no standard train/test split for this dataset but you can easily split it according to year, language pair or domain. E.g. :



Note that most data is from News domain.




If you use this data please cite the WMT findings from previous years:
- Findings of the 2017 Conference on Machine Translation (WMT17)
- Findings of the 2018 Conference on Machine Translation (WMT18)
- Findings of the 2019 Conference on Machine Translation (WMT19)
- Findings of the 2020 Conference on Machine Translation (WMT20)
- Findings of the 2021 Conference on Machine Translation (WMT21)
- Findings of the 2022 Conference on Machine Translation (WMT22)