
# Description
Dataset used to train our mT5 based model for machine translation, extracted from various text sources of National Library of Azerbaijan: [mT5-translation-enaz](https://huggingface.co/learningmachineaz/mt5-enaz-10m) \
It has only clean texts. Wiki articles wasn't used as they contain a lot of irrelevant data.

| Key point               | Info |
|-------------------------|---------|
| Rows | ~10mil. EN-AZ sentence pairs    |
| Size | 975M (zipped) / 2.8G (unzipped)    |
| Format | TSV (tab separated pairs)    |
| English | Google Translate    |
| Azerbaijani | Original cleaned text    |

## Author
Collected and prepared by [Renat Kalimulin](https://www.linkedin.com/in/rinat-kalimulin-16853358/)

task_categories:translation task_categories:text-generation task_categories:text2text-generation size_categories:1M<n<10M language:en language:az license:openrail azerbaijani books azerbaijani news azerbaijani poems azerbaijani articles azerbaijani dataset region:us
{"language": ["en", "az"], "license": "openrail", "size_categories": ["1M<n<10M"], "task_categories": ["translation", "text-generation", "text2text-generation"], "pretty_name": "English-Azerbaijani Dataset", "tags": ["azerbaijani books", "azerbaijani news", "azerbaijani poems", "azerbaijani articles", "azerbaijani dataset"]}
TAGS
#task_categories-translation #task_categories-text-generation #task_categories-text2text-generation #size_categories-1M<n<10M #language-English #language-Azerbaijani #license-openrail #azerbaijani books #azerbaijani news #azerbaijani poems #azerbaijani articles #azerbaijani dataset #region-us 

Description
===========


Dataset used to train our mT5 based model for machine translation, extracted from various text sources of National Library of Azerbaijan: mT5-translation-enaz   

It has only clean texts. Wiki articles wasn't used as they contain a lot of irrelevant data.



Author
------


Collected and prepared by Renat Kalimulin



TAGS
#task_categories-translation #task_categories-text-generation #task_categories-text2text-generation #size_categories-1M<n<10M #language-English #language-Azerbaijani #license-openrail #azerbaijani books #azerbaijani news #azerbaijani poems #azerbaijani articles #azerbaijani dataset #region-us 
