
# Dataset Card for MultiUN

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** https://opus.nlpl.eu/MultiUN/corpus/version/MultiUN
- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Paper:** https://aclanthology.org/L10-1473/
- **Leaderboard:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

### Dataset Summary

The MultiUN parallel corpus is extracted from the United Nations Website , and then cleaned and converted to XML at Language Technology Lab in DFKI GmbH (LT-DFKI), Germany. The documents were published by UN from 2000 to 2009.

This is a collection of translated documents from the United Nations originally compiled by Andreas Eisele and Yu Chen (see http://www.euromatrixplus.net/multi-un/).

This corpus is available in all 6 official languages of the UN consisting of around 300 million words per language

### Supported Tasks and Leaderboards

The underlying task is machine translation.

### Languages

Parallel texts are present in all six official languages, namely Arabic, Chinese,
English, French, Russian and Spanish, with a small part of the documents available also in German.

## Dataset Structure

### Data Instances

[More Information Needed]

### Data Fields

[More Information Needed]

### Data Splits

[More Information Needed]

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

Original MultiUN source data: http://www.euromatrixplus.net/multi-unp

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

If you use this corpus in your work, please cite the paper:
```
@inproceedings{eisele-chen-2010-multiun,
    title = "{M}ulti{UN}: A Multilingual Corpus from United Nation Documents",
    author = "Eisele, Andreas  and
      Chen, Yu",
    booktitle = "Proceedings of the Seventh International Conference on Language Resources and Evaluation ({LREC}'10)",
    month = may,
    year = "2010",
    address = "Valletta, Malta",
    publisher = "European Language Resources Association (ELRA)",
    url = "http://www.lrec-conf.org/proceedings/lrec2010/pdf/686_Paper.pdf",
    abstract = "This paper describes the acquisition, preparation and properties of a corpus extracted from the official documents of the United Nations (UN). This corpus is available in all 6 official languages of the UN, consisting of around 300 million words per language. We describe the methods we used for crawling, document formatting, and sentence alignment. This corpus also includes a common test set for machine translation. We present the results of a French-Chinese machine translation experiment performed on this corpus.",
}
```

If you use any part of the corpus (hosted in OPUS) in your own work, please cite the following article:
```
@inproceedings{tiedemann-2012-parallel,
    title = "Parallel Data, Tools and Interfaces in {OPUS}",
    author = {Tiedemann, J{\"o}rg},
    editor = "Calzolari, Nicoletta  and
      Choukri, Khalid  and
      Declerck, Thierry  and
      Do{\u{g}}an, Mehmet U{\u{g}}ur  and
      Maegaard, Bente  and
      Mariani, Joseph  and
      Moreno, Asuncion  and
      Odijk, Jan  and
      Piperidis, Stelios",
    booktitle = "Proceedings of the Eighth International Conference on Language Resources and Evaluation ({LREC}'12)",
    month = may,
    year = "2012",
    address = "Istanbul, Turkey",
    publisher = "European Language Resources Association (ELRA)",
    url = "http://www.lrec-conf.org/proceedings/lrec2012/pdf/463_Paper.pdf",
    pages = "2214--2218",
    abstract = "This paper presents the current status of OPUS, a growing language resource of parallel corpora and related tools. The focus in OPUS is to provide freely available data sets in various formats together with basic annotation to be useful for applications in computational linguistics, translation studies and cross-linguistic corpus studies. In this paper, we report about new data sets and their features, additional annotation tools and models provided from the website and essential interfaces and on-line services included in the project.",
}
```

### Contributions

Thanks to [@patil-suraj](https://github.com/patil-suraj) for adding this dataset.
task_categories:translation annotations_creators:found language_creators:found multilinguality:multilingual size_categories:100K<n<1M source_datasets:original language:ar language:de language:en language:es language:fr language:ru language:zh license:unknown region:us
{"annotations_creators": ["found"], "language_creators": ["found"], "language": ["ar", "de", "en", "es", "fr", "ru", "zh"], "license": ["unknown"], "multilinguality": ["multilingual"], "size_categories": ["100K<n<1M"], "source_datasets": ["original"], "task_categories": ["translation"], "task_ids": [], "paperswithcode_id": "multiun", "pretty_name": "Multilingual Corpus from United Nation Documents", "config_names": ["ar-de", "ar-en", "ar-es", "ar-fr", "ar-ru", "ar-zh", "de-en", "de-es", "de-fr", "de-ru", "de-zh", "en-es", "en-fr", "en-ru", "en-zh", "es-fr", "es-ru", "es-zh", "fr-ru", "fr-zh", "ru-zh"], "dataset_info": [{"config_name": "ar-de", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["ar", "de"]}}}], "splits": [{"name": "train", "num_bytes": 94466397, "num_examples": 165090}], "download_size": 21869935, "dataset_size": 94466397}, {"config_name": "ar-en", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["ar", "en"]}}}], "splits": [{"name": "train", "num_bytes": 4189852369, "num_examples": 9759125}], "download_size": 1036296368, "dataset_size": 4189852369}, {"config_name": "ar-es", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["ar", "es"]}}}], "splits": [{"name": "train", "num_bytes": 4509675284, "num_examples": 10119379}], "download_size": 1101206667, "dataset_size": 4509675284}, {"config_name": "ar-fr", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["ar", "fr"]}}}], "splits": [{"name": "train", "num_bytes": 4516850009, "num_examples": 9929567}], "download_size": 1109705925, "dataset_size": 4516850009}, {"config_name": "ar-ru", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["ar", "ru"]}}}], "splits": [{"name": "train", "num_bytes": 5932866867, "num_examples": 10206243}], "download_size": 1261123878, "dataset_size": 5932866867}, {"config_name": "ar-zh", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["ar", "zh"]}}}], "splits": [{"name": "train", "num_bytes": 3781658413, "num_examples": 9832293}], "download_size": 1009696775, "dataset_size": 3781658413}, {"config_name": "de-en", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["de", "en"]}}}], "splits": [{"name": "train", "num_bytes": 76684549, "num_examples": 162981}], "download_size": 19468529, "dataset_size": 76684549}, {"config_name": "de-es", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["de", "es"]}}}], "splits": [{"name": "train", "num_bytes": 80936653, "num_examples": 162078}], "download_size": 20266591, "dataset_size": 80936653}, {"config_name": "de-fr", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["de", "fr"]}}}], "splits": [{"name": "train", "num_bytes": 81888435, "num_examples": 164025}], "download_size": 20692837, "dataset_size": 81888435}, {"config_name": "de-ru", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["de", "ru"]}}}], "splits": [{"name": "train", "num_bytes": 111517934, "num_examples": 164792}], "download_size": 23507789, "dataset_size": 111517934}, {"config_name": "de-zh", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["de", "zh"]}}}], "splits": [{"name": "train", "num_bytes": 70534818, "num_examples": 176933}], "download_size": 19927209, "dataset_size": 70534818}, {"config_name": "en-es", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["en", "es"]}}}], "splits": [{"name": "train", "num_bytes": 4128141663, "num_examples": 11350967}], "download_size": 1123164180, "dataset_size": 4128141663}, {"config_name": "en-fr", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["en", "fr"]}}}], "splits": [{"name": "train", "num_bytes": 4678055160, "num_examples": 13172019}], "download_size": 1355002731, "dataset_size": 4678055160}, {"config_name": "en-ru", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["en", "ru"]}}}], "splits": [{"name": "train", "num_bytes": 5632662839, "num_examples": 11654416}], "download_size": 1285801078, "dataset_size": 5632662839}, {"config_name": "en-zh", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["en", "zh"]}}}], "splits": [{"name": "train", "num_bytes": 2960376046, "num_examples": 9564315}], "download_size": 900076520, "dataset_size": 2960376046}, {"config_name": "es-fr", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["es", "fr"]}}}], "splits": [{"name": "train", "num_bytes": 4454712498, "num_examples": 11441889}], "download_size": 1195733510, "dataset_size": 4454712498}, {"config_name": "es-ru", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["es", "ru"]}}}], "splits": [{"name": "train", "num_bytes": 5442655730, "num_examples": 10605056}], "download_size": 1228045966, "dataset_size": 5442655730}, {"config_name": "es-zh", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["es", "zh"]}}}], "splits": [{"name": "train", "num_bytes": 3223871198, "num_examples": 9847770}], "download_size": 953250084, "dataset_size": 3223871198}, {"config_name": "fr-ru", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["fr", "ru"]}}}], "splits": [{"name": "train", "num_bytes": 5979879089, "num_examples": 11761738}], "download_size": 1364307157, "dataset_size": 5979879089}, {"config_name": "fr-zh", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["fr", "zh"]}}}], "splits": [{"name": "train", "num_bytes": 3241098333, "num_examples": 9690914}], "download_size": 962824881, "dataset_size": 3241098333}, {"config_name": "ru-zh", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["ru", "zh"]}}}], "splits": [{"name": "train", "num_bytes": 4233875537, "num_examples": 9557007}], "download_size": 1037881127, "dataset_size": 4233875537}]}
TAGS
#task_categories-translation #annotations_creators-found #language_creators-found #multilinguality-multilingual #size_categories-100K<n<1M #source_datasets-original #language-Arabic #language-German #language-English #language-Spanish #language-French #language-Russian #language-Chinese #license-unknown #region-us 


# Dataset Card for MultiUN

## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions

## Dataset Description

- Homepage: URL
- Repository: 
- Paper: URL
- Leaderboard: 
- Point of Contact: 

### Dataset Summary

The MultiUN parallel corpus is extracted from the United Nations Website , and then cleaned and converted to XML at Language Technology Lab in DFKI GmbH (LT-DFKI), Germany. The documents were published by UN from 2000 to 2009.

This is a collection of translated documents from the United Nations originally compiled by Andreas Eisele and Yu Chen (see URL

This corpus is available in all 6 official languages of the UN consisting of around 300 million words per language

### Supported Tasks and Leaderboards

The underlying task is machine translation.

### Languages

Parallel texts are present in all six official languages, namely Arabic, Chinese,
English, French, Russian and Spanish, with a small part of the documents available also in German.

## Dataset Structure

### Data Instances



### Data Fields



### Data Splits



## Dataset Creation

### Curation Rationale



### Source Data

#### Initial Data Collection and Normalization

Original MultiUN source data: URL

#### Who are the source language producers?



### Annotations

#### Annotation process



#### Who are the annotators?



### Personal and Sensitive Information



## Considerations for Using the Data

### Social Impact of Dataset



### Discussion of Biases



### Other Known Limitations



## Additional Information

### Dataset Curators



### Licensing Information





If you use this corpus in your work, please cite the paper:


If you use any part of the corpus (hosted in OPUS) in your own work, please cite the following article:


### Contributions

Thanks to @patil-suraj for adding this dataset.
# Dataset Card for MultiUN ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions ## Dataset Description

- Homepage: URL
- Repository: 
- Paper: URL
- Leaderboard: 
- Point of Contact: ### Dataset Summary

The MultiUN parallel corpus is extracted from the United Nations Website , and then cleaned and converted to XML at Language Technology Lab in DFKI GmbH (LT-DFKI), Germany. The documents were published by UN from 2000 to 2009.

This is a collection of translated documents from the United Nations originally compiled by Andreas Eisele and Yu Chen (see URL

This corpus is available in all 6 official languages of the UN consisting of around 300 million words per language ### Supported Tasks and Leaderboards

The underlying task is machine translation. ### Languages

Parallel texts are present in all six official languages, namely Arabic, Chinese,
English, French, Russian and Spanish, with a small part of the documents available also in German. ## Dataset Structure ### Data Instances ### Data Fields ### Data Splits ## Dataset Creation ### Curation Rationale ### Source Data #### Initial Data Collection and Normalization

Original MultiUN source data: URL #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations ## Additional Information ### Dataset Curators ### Licensing Information





If you use this corpus in your work, please cite the paper:


If you use any part of the corpus (hosted in OPUS) in your own work, please cite the following article: ### Contributions

Thanks to @patil-suraj for adding this dataset.
TAGS
#task_categories-translation #annotations_creators-found #language_creators-found #multilinguality-multilingual #size_categories-100K<n<1M #source_datasets-original #language-Arabic #language-German #language-English #language-Spanish #language-French #language-Russian #language-Chinese #license-unknown #region-us 
 # Dataset Card for MultiUN ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Supported Tasks and Leaderboards
  - Languages
- Dataset Structure
  - Data Instances
  - Data Fields
  - Data Splits
- Dataset Creation
  - Curation Rationale
  - Source Data
  - Annotations
  - Personal and Sensitive Information
- Considerations for Using the Data
  - Social Impact of Dataset
  - Discussion of Biases
  - Other Known Limitations
- Additional Information
  - Dataset Curators
  - Licensing Information
  - Citation Information
  - Contributions ## Dataset Description

- Homepage: URL
- Repository: 
- Paper: URL
- Leaderboard: 
- Point of Contact: ### Dataset Summary

The MultiUN parallel corpus is extracted from the United Nations Website , and then cleaned and converted to XML at Language Technology Lab in DFKI GmbH (LT-DFKI), Germany. The documents were published by UN from 2000 to 2009.

This is a collection of translated documents from the United Nations originally compiled by Andreas Eisele and Yu Chen (see URL

This corpus is available in all 6 official languages of the UN consisting of around 300 million words per language ### Supported Tasks and Leaderboards

The underlying task is machine translation. ### Languages

Parallel texts are present in all six official languages, namely Arabic, Chinese,
English, French, Russian and Spanish, with a small part of the documents available also in German. ## Dataset Structure ### Data Instances ### Data Fields ### Data Splits ## Dataset Creation ### Curation Rationale ### Source Data #### Initial Data Collection and Normalization

Original MultiUN source data: URL #### Who are the source language producers? ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information ## Considerations for Using the Data ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations ## Additional Information ### Dataset Curators ### Licensing Information





If you use this corpus in your work, please cite the paper:


If you use any part of the corpus (hosted in OPUS) in your own work, please cite the following article: ### Contributions

Thanks to @patil-suraj for adding this dataset.