
# Dataset Card for poleval2019_mt

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** PolEval-2019 competition. http://2019.poleval.pl/
- **Repository:** Links available [in this page](http://2019.poleval.pl/index.php/tasks/task4)
- **Paper:**
- **Leaderboard:**
- **Point of Contact:**

### Dataset Summary

PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish.
Submitted solutions compete against one another within certain tasks selected by organizers, using available data and are evaluated according to
pre-established procedures. One of the tasks in PolEval-2019 was Machine Translation (Task-4).

The task is to train as good as possible machine translation system, using any technology,with limited textual resources.
The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced
Russian-Polish (in both directions).

Here, Polish-English is also made available to allow for training in both directions. However, the test data is ONLY available for English-Polish

### Supported Tasks and Leaderboards
Supports Machine Translation between Russian to Polish and English to Polish (and vice versa). 

### Languages
- Polish (pl)
- Russian (ru)
- English (en) 

## Dataset Structure

### Data Instances
As the training data set, a set of bi-lingual corpora aligned at the sentence level has been prepared. The corpora are saved in UTF-8 encoding as plain text, one language per file.

### Data Fields
One example of the translation is as below:
```
{
  'translation': {'ru': 'не содержала в себе моделей. Модели это сравнительно новое явление. ', 
                  'pl': 'nie miała w sobie modeli. Modele to względnie nowa dziedzina. Tak więc, jeśli '}
}
```

### Data Splits

The dataset is divided into two splits. All the headlines are scraped from news websites on the internet.

|       |  train | validation | test |
|-------|-------:|-----------:|-----:|
| ru-pl |  20001 |       3001 | 2969 |
| pl-ru |  20001 |       3001 | 2969 |
| en-pl | 129255 |       1000 | 9845 |

## Dataset Creation

### Curation Rationale

This data was curated as a task for the PolEval-2019. The task is to train as good as possible machine translation system, using any technology, with limited textual resources. The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced Russian-Polish (in both directions).

PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish. Submitted tools compete against one another within certain tasks selected by organizers, using available data and are evaluated according to pre-established procedures.

PolEval 2019-related papers were presented at AI & NLP Workshop Day (Warsaw, May 31, 2019).
The links for the top performing models on various tasks (including the Task-4: Machine Translation) is present in [this](http://2019.poleval.pl/index.php/publication) link

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

The organization details of PolEval is present in this [link](http://2019.poleval.pl/index.php/organizers)

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

```
@proceedings{ogr:kob:19:poleval,
  editor    = {Maciej Ogrodniczuk and Łukasz Kobyliński},
  title     = {{Proceedings of the PolEval 2019 Workshop}},
  year      = {2019},
  address   = {Warsaw, Poland},
  publisher = {Institute of Computer Science, Polish Academy of Sciences},
  url       = {http://2019.poleval.pl/files/poleval2019.pdf},
  isbn      = "978-83-63159-28-3"}
}
```

### Contributions

Thanks to [@vrindaprabhu](https://github.com/vrindaprabhu) for adding this dataset.
task_categories:translation annotations_creators:no-annotation language_creators:expert-generated language_creators:found multilinguality:translation size_categories:10K<n<100K source_datasets:original language:en language:pl language:ru license:unknown region:us
{"annotations_creators": ["no-annotation"], "language_creators": ["expert-generated", "found"], "language": ["en", "pl", "ru"], "license": ["unknown"], "multilinguality": ["translation"], "size_categories": ["10K<n<100K"], "source_datasets": ["original"], "task_categories": ["translation"], "task_ids": [], "pretty_name": "Poleval2019Mt", "dataset_info": [{"config_name": "ru-pl", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["ru", "pl"]}}}], "splits": [{"name": "train", "num_bytes": 2818015, "num_examples": 20001}, {"name": "validation", "num_bytes": 415735, "num_examples": 3001}, {"name": "test", "num_bytes": 266462, "num_examples": 2969}], "download_size": 3355801, "dataset_size": 3500212}, {"config_name": "en-pl", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["en", "pl"]}}}], "splits": [{"name": "train", "num_bytes": 13217798, "num_examples": 129255}, {"name": "validation", "num_bytes": 1209168, "num_examples": 10001}, {"name": "test", "num_bytes": 562482, "num_examples": 9845}], "download_size": 13851405, "dataset_size": 14989448}, {"config_name": "pl-ru", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["pl", "ru"]}}}], "splits": [{"name": "train", "num_bytes": 2818015, "num_examples": 20001}, {"name": "validation", "num_bytes": 415735, "num_examples": 3001}, {"name": "test", "num_bytes": 149423, "num_examples": 2967}], "download_size": 3355801, "dataset_size": 3383173}, {"config_name": "pl-en", "features": [{"name": "translation", "dtype": {"translation": {"languages": ["pl", "en"]}}}], "splits": [{"name": "train", "num_bytes": 13217798, "num_examples": 129255}, {"name": "validation", "num_bytes": 1209168, "num_examples": 10001}, {"name": "test", "num_bytes": 16, "num_examples": 1}], "download_size": 13591306, "dataset_size": 14426982}]}
TAGS
#task_categories-translation #annotations_creators-no-annotation #language_creators-expert-generated #language_creators-found #multilinguality-translation #size_categories-10K<n<100K #source_datasets-original #language-English #language-Polish #language-Russian #license-unknown #region-us 

Dataset Card for poleval2019\_mt
================================


Table of Contents
-----------------


* Dataset Description
	+ Dataset Summary
	+ Supported Tasks and Leaderboards
	+ Languages
* Dataset Structure
	+ Data Instances
	+ Data Fields
	+ Data Splits
* Dataset Creation
	+ Curation Rationale
	+ Source Data
	+ Annotations
	+ Personal and Sensitive Information
* Considerations for Using the Data
	+ Social Impact of Dataset
	+ Discussion of Biases
	+ Other Known Limitations
* Additional Information
	+ Dataset Curators
	+ Licensing Information
	+ Citation Information
	+ Contributions


Dataset Description
-------------------


* Homepage: PolEval-2019 competition. URL
* Repository: Links available in this page
* Paper:
* Leaderboard:
* Point of Contact:


### Dataset Summary


PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish.
Submitted solutions compete against one another within certain tasks selected by organizers, using available data and are evaluated according to
pre-established procedures. One of the tasks in PolEval-2019 was Machine Translation (Task-4).


The task is to train as good as possible machine translation system, using any technology,with limited textual resources.
The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced
Russian-Polish (in both directions).


Here, Polish-English is also made available to allow for training in both directions. However, the test data is ONLY available for English-Polish


### Supported Tasks and Leaderboards


Supports Machine Translation between Russian to Polish and English to Polish (and vice versa).


### Languages


* Polish (pl)
* Russian (ru)
* English (en)


Dataset Structure
-----------------


### Data Instances


As the training data set, a set of bi-lingual corpora aligned at the sentence level has been prepared. The corpora are saved in UTF-8 encoding as plain text, one language per file.


### Data Fields


One example of the translation is as below:


### Data Splits


The dataset is divided into two splits. All the headlines are scraped from news websites on the internet.



Dataset Creation
----------------


### Curation Rationale


This data was curated as a task for the PolEval-2019. The task is to train as good as possible machine translation system, using any technology, with limited textual resources. The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced Russian-Polish (in both directions).


PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish. Submitted tools compete against one another within certain tasks selected by organizers, using available data and are evaluated according to pre-established procedures.


PolEval 2019-related papers were presented at AI & NLP Workshop Day (Warsaw, May 31, 2019).
The links for the top performing models on various tasks (including the Task-4: Machine Translation) is present in this link


### Source Data


#### Initial Data Collection and Normalization


#### Who are the source language producers?


The organization details of PolEval is present in this link


### Annotations


#### Annotation process


#### Who are the annotators?


### Personal and Sensitive Information


Considerations for Using the Data
---------------------------------


### Social Impact of Dataset


### Discussion of Biases


### Other Known Limitations


Additional Information
----------------------


### Dataset Curators


### Licensing Information


### Contributions


Thanks to @vrindaprabhu for adding this dataset.



### Dataset Summary


PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish.
Submitted solutions compete against one another within certain tasks selected by organizers, using available data and are evaluated according to
pre-established procedures. One of the tasks in PolEval-2019 was Machine Translation (Task-4).


The task is to train as good as possible machine translation system, using any technology,with limited textual resources.
The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced
Russian-Polish (in both directions).


Here, Polish-English is also made available to allow for training in both directions. However, the test data is ONLY available for English-Polish ### Supported Tasks and Leaderboards


Supports Machine Translation between Russian to Polish and English to Polish (and vice versa). ### Languages


* Polish (pl)
* Russian (ru)
* English (en)


Dataset Structure
----------------- ### Data Instances


As the training data set, a set of bi-lingual corpora aligned at the sentence level has been prepared. The corpora are saved in UTF-8 encoding as plain text, one language per file. ### Data Fields


One example of the translation is as below: ### Data Splits


The dataset is divided into two splits. All the headlines are scraped from news websites on the internet.



Dataset Creation
---------------- ### Curation Rationale


This data was curated as a task for the PolEval-2019. The task is to train as good as possible machine translation system, using any technology, with limited textual resources. The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced Russian-Polish (in both directions).


PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish. Submitted tools compete against one another within certain tasks selected by organizers, using available data and are evaluated according to pre-established procedures.


PolEval 2019-related papers were presented at AI & NLP Workshop Day (Warsaw, May 31, 2019).
The links for the top performing models on various tasks (including the Task-4: Machine Translation) is present in this link ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers?


The organization details of PolEval is present in this link ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators ### Licensing Information ### Contributions


Thanks to @vrindaprabhu for adding this dataset.
TAGS
#task_categories-translation #annotations_creators-no-annotation #language_creators-expert-generated #language_creators-found #multilinguality-translation #size_categories-10K<n<100K #source_datasets-original #language-English #language-Polish #language-Russian #license-unknown #region-us 
 ### Dataset Summary


PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish.
Submitted solutions compete against one another within certain tasks selected by organizers, using available data and are evaluated according to
pre-established procedures. One of the tasks in PolEval-2019 was Machine Translation (Task-4).


The task is to train as good as possible machine translation system, using any technology,with limited textual resources.
The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced
Russian-Polish (in both directions).


Here, Polish-English is also made available to allow for training in both directions. However, the test data is ONLY available for English-Polish ### Supported Tasks and Leaderboards


Supports Machine Translation between Russian to Polish and English to Polish (and vice versa). ### Languages


* Polish (pl)
* Russian (ru)
* English (en)


Dataset Structure
----------------- ### Data Instances


As the training data set, a set of bi-lingual corpora aligned at the sentence level has been prepared. The corpora are saved in UTF-8 encoding as plain text, one language per file. ### Data Fields


One example of the translation is as below: ### Data Splits


The dataset is divided into two splits. All the headlines are scraped from news websites on the internet.



Dataset Creation
---------------- ### Curation Rationale


This data was curated as a task for the PolEval-2019. The task is to train as good as possible machine translation system, using any technology, with limited textual resources. The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced Russian-Polish (in both directions).


PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish. Submitted tools compete against one another within certain tasks selected by organizers, using available data and are evaluated according to pre-established procedures.


PolEval 2019-related papers were presented at AI & NLP Workshop Day (Warsaw, May 31, 2019).
The links for the top performing models on various tasks (including the Task-4: Machine Translation) is present in this link ### Source Data #### Initial Data Collection and Normalization #### Who are the source language producers?


The organization details of PolEval is present in this link ### Annotations #### Annotation process #### Who are the annotators? ### Personal and Sensitive Information


Considerations for Using the Data
--------------------------------- ### Social Impact of Dataset ### Discussion of Biases ### Other Known Limitations


Additional Information
---------------------- ### Dataset Curators ### Licensing Information ### Contributions


Thanks to @vrindaprabhu for adding this dataset.
passage: TAGS
#task_categories-translation #annotations_creators-no-annotation #language_creators-expert-generated #language_creators-found #multilinguality-translation #size_categories-10K<n<100K #source_datasets-original #language-English #language-Polish #language-Russian #license-unknown #region-us 
### Dataset Summary


PolEval is a SemEval-inspired evaluation campaign for natural language processing tools for Polish.
Submitted solutions compete against one another within certain tasks selected by organizers, using available data and are evaluated according to
pre-established procedures. One of the tasks in PolEval-2019 was Machine Translation (Task-4).


The task is to train as good as possible machine translation system, using any technology,with limited textual resources.
The competition will be done for 2 language pairs, more popular English-Polish (into Polish direction) and pair that can be called low resourced
Russian-Polish (in both directions).


Here, Polish-English is also made available to allow for training in both directions. However, the test data is ONLY available for English-Polish### Supported Tasks and Leaderboards


Supports Machine Translation between Russian to Polish and English to Polish (and vice versa).### Languages


* Polish (pl)
* Russian (ru)
* English (en)


Dataset Structure
-----------------### Data Instances


As the training data set, a set of bi-lingual corpora aligned at the sentence level has been prepared. The corpora are saved in UTF-8 encoding as plain text, one language per file.### Data Fields


One example of the translation is as below:### Data Splits


The dataset is divided into two splits. All the headlines are scraped from news websites on the internet.



Dataset Creation
----------------