
# Dataset Card for Wikipedia

This repo is a wrapper around [olm/wikipedia](https://huggingface.co/datasets/olm/wikipedia) that just concatenates data from the EU languages.
Please refer to it for a complete data card.

The EU languages we include are:
   - bg
   - cs
   - da
   - de
   - el
   - en
   - es
   - et
   - fi
   - fr
   - ga
   - hr
   - hu
   - it
   - lt
   - lv
   - mt
   - nl
   - pl
   - pt
   - ro
   - sk
   - sl
   - sv


As with `olm/wikipedia` you will need to install a few dependencies:


```
pip install mwparserfromhell==0.6.4 multiprocess==0.70.13
```

```python
from datasets import load_dataset

load_dataset("dlwh/eu_wikipedias", date="20221101")
```

Please refer to the original olm/wikipedia for a complete data card.

task_categories:text-generation task_categories:fill-mask task_ids:language-modeling task_ids:masked-language-modeling annotations_creators:no-annotation language_creators:crowdsourced multilinguality:multilingual size_categories:n<1K size_categories:1K<n<10K size_categories:10K<n<100K size_categories:100K<n<1M size_categories:1M<n<10M source_datasets:original language:bg language:cs language:da language:de language:el language:en language:es language:et language:fi language:fr language:ga language:hr language:hu language:it language:lt language:lv language:mt language:nl language:pl language:pt language:ro language:sk language:sl language:sv license:cc-by-sa-3.0 license:gfdl region:us
{"annotations_creators": ["no-annotation"], "language_creators": ["crowdsourced"], "language": ["bg", "cs", "da", "de", "el", "en", "es", "et", "fi", "fr", "ga", "hr", "hu", "it", "lt", "lv", "mt", "nl", "pl", "pt", "ro", "sk", "sl", "sv"], "license": ["cc-by-sa-3.0", "gfdl"], "multilinguality": ["multilingual"], "size_categories": ["n<1K", "1K<n<10K", "10K<n<100K", "100K<n<1M", "1M<n<10M"], "source_datasets": ["original"], "task_categories": ["text-generation", "fill-mask"], "task_ids": ["language-modeling", "masked-language-modeling"], "pretty_name": "Wikipedia"}
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-crowdsourced #multilinguality-multilingual #size_categories-n<1K #size_categories-1K<n<10K #size_categories-10K<n<100K #size_categories-100K<n<1M #size_categories-1M<n<10M #source_datasets-original #language-Bulgarian #language-Czech #language-Danish #language-German #language-Modern Greek (1453-) #language-English #language-Spanish #language-Estonian #language-Finnish #language-French #language-Irish #language-Croatian #language-Hungarian #language-Italian #language-Lithuanian #language-Latvian #language-Maltese #language-Dutch #language-Polish #language-Portuguese #language-Romanian #language-Slovak #language-Slovenian #language-Swedish #license-cc-by-sa-3.0 #license-gfdl #region-us 


# Dataset Card for Wikipedia

This repo is a wrapper around olm/wikipedia that just concatenates data from the EU languages.
Please refer to it for a complete data card.

The EU languages we include are:
   - bg
   - cs
   - da
   - de
   - el
   - en
   - es
   - et
   - fi
   - fr
   - ga
   - hr
   - hu
   - it
   - lt
   - lv
   - mt
   - nl
   - pl
   - pt
   - ro
   - sk
   - sl
   - sv


As with 'olm/wikipedia' you will need to install a few dependencies:






Please refer to the original olm/wikipedia for a complete data card.

# Dataset Card for Wikipedia

This repo is a wrapper around olm/wikipedia that just concatenates data from the EU languages.
Please refer to it for a complete data card.

The EU languages we include are:
   - bg
   - cs
   - da
   - de
   - el
   - en
   - es
   - et
   - fi
   - fr
   - ga
   - hr
   - hu
   - it
   - lt
   - lv
   - mt
   - nl
   - pl
   - pt
   - ro
   - sk
   - sl
   - sv


As with 'olm/wikipedia' you will need to install a few dependencies:






Please refer to the original olm/wikipedia for a complete data card.
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-crowdsourced #multilinguality-multilingual #size_categories-n<1K #size_categories-1K<n<10K #size_categories-10K<n<100K #size_categories-100K<n<1M #size_categories-1M<n<10M #source_datasets-original #language-Bulgarian #language-Czech #language-Danish #language-German #language-Modern Greek (1453-) #language-English #language-Spanish #language-Estonian #language-Finnish #language-French #language-Irish #language-Croatian #language-Hungarian #language-Italian #language-Lithuanian #language-Latvian #language-Maltese #language-Dutch #language-Polish #language-Portuguese #language-Romanian #language-Slovak #language-Slovenian #language-Swedish #license-cc-by-sa-3.0 #license-gfdl #region-us 
 # Dataset Card for Wikipedia

This repo is a wrapper around olm/wikipedia that just concatenates data from the EU languages.
Please refer to it for a complete data card.

The EU languages we include are:
   - bg
   - cs
   - da
   - de
   - el
   - en
   - es
   - et
   - fi
   - fr
   - ga
   - hr
   - hu
   - it
   - lt
   - lv
   - mt
   - nl
   - pl
   - pt
   - ro
   - sk
   - sl
   - sv


As with 'olm/wikipedia' you will need to install a few dependencies:






Please refer to the original olm/wikipedia for a complete data card.
passage: TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-crowdsourced #multilinguality-multilingual #size_categories-n<1K #size_categories-1K<n<10K #size_categories-10K<n<100K #size_categories-100K<n<1M #size_categories-1M<n<10M #source_datasets-original #language-Bulgarian #language-Czech #language-Danish #language-German #language-Modern Greek (1453-) #language-English #language-Spanish #language-Estonian #language-Finnish #language-French #language-Irish #language-Croatian #language-Hungarian #language-Italian #language-Lithuanian #language-Latvian #language-Maltese #language-Dutch #language-Polish #language-Portuguese #language-Romanian #language-Slovak #language-Slovenian #language-Swedish #license-cc-by-sa-3.0 #license-gfdl #region-us 
# Dataset Card for Wikipedia

This repo is a wrapper around olm/wikipedia that just concatenates data from the EU languages.
Please refer to it for a complete data card.

The EU languages we include are:
   - bg
   - cs
   - da
   - de
   - el
   - en
   - es
   - et
   - fi
   - fr
   - ga
   - hr
   - hu
   - it
   - lt
   - lv
   - mt
   - nl
   - pl
   - pt
   - ro
   - sk
   - sl
   - sv


As with 'olm/wikipedia' you will need to install a few dependencies:






Please refer to the original olm/wikipedia for a complete data card.