
# Wikipedia

This Wikipedia dataset contains all available languages for recent dumps. It is
a refresh of the [20220301 wikipedia](https://hf.co/datasets/wikipedia) from
Huggingface, so it has the same license and dataset card details. The benefits
of this dataset are:

- more recent dumps (see table below)
- a few additional languages
- all available languages are preprocessed (including the largests: `en` and
  `ceb`)

| version | dump     | # available languages               | closed & dump                                   | closed & no dump  |
| -----   | ----     | -----                               | ------                                          | ---               |
| `1.0.0` | 20230601 | 328                                 | 9: ak (soon), cho, ho, ii, kj, lrc, mh, mus, ng | 4: aa, hz, kr, na |
| `1.1.0` | 20230601 | 329 (+et ~[az,ceb,ch,hr,ii,lrc,ta]) | 9: ak (soon), cho, ho, ii, kj, lrc, mh, mus, ng | 4: aa, hz, kr, na |
| `1.2.0` | 20230901 | idem                                | 9: ak , cho, ho, ii, kj, lrc, mh, mus, ng       | 4: aa, hz, kr, na |

Source: [List of Wikimedia
Languages](https://en.wikipedia.org/wiki/List_of_Wikipedias). A few (9)
Wikimedias are closed, meaning they won't have new pages, but the dumps are
still available. In addition, very few (4) Wikimedias are closed and don't
have dumps anymore.

## Release Notes

`1.2.0`

- **chore**: Update to 20230901

`1.1.0`

- **feat**: Add missing estonian (my bad), thanks Chris Ha
- **fix**: update category lists for az, ceb, ch, hr, ii, lrc, ta, which means
  they were all processed again.

`1.0.0`

- **chore**: File layout is now `data/{dump}/{lang}/{info.json,*.parquet}`.
  Sorry for the radical update, probably won't happen again.
- **chore**: Parquet files are now sharded (size < 200 MB), allowing parallel
  downloads and processing.
- **fix**: All languages were all processed again because of a bug in the media
  and category names, leading to some links not being extracted.
- **feat**: Add `en` and `ceb` which were too big for my Beam DirectRunner at
  the time.

## Usage

```python
from datasets import load_dataset

wikipedia_es = load_dataset("graelo/wikipedia", "20230601.es")
```

---

## Build instructions

Developer only. This dataset was preprocessed with a Beam DirectRunner as
follows.

### 1. Determine the date of the dump you are interested in

Choose one wikipedia dump, for instance <https://dumps.wikimedia.org/cewiki/>
and identify the date.

### 2. [Optional] Get a refreshed list of languages

This is optional because it not very likely that a new language will have
suddenly appeared since the last version _and_ have a significant dataset.

Navigate to <https://en.wikipedia.org/wiki/List_of_Wikipedias> and copy the
languages column from the "Detailed list" table (near the end of the page).

Copy that content in the form of a Python list into `lang_def.py` (at the top
of the repo) under a new date.

### 3. [Optional] Create Media and Category aliases

In order to properly extract links to images and media in all languages, we
must refresh the two corresponding files. To do so, from the root of the repo,
run

```sh
python -m prep.create_aliases
```

This will create or update these two files at the root of the repo:

- `media_aliases.py`
- `category_aliases.py`

These files are used in the final step

### 4. Build and prepare the datasets into sharded parquet files

Running this script downloads the wikipedia dumps for each language in
`lang_def.py` and shards each language dataset into the appropriate number of
shards (max size ~ 250MB).

```sh
python -m prep.build --date 20230601
```

There are other options:

```text
$ python -m prep.build --help
usage: Wikipedia Builder [-h] [--date DATE] [--language [LANG ...]] [--cache-dir DIR] [--mirror MIRROR]

Prepares the Wikipedia dataset for each language

optional arguments:
  -h, --help             show this help message and exit
  --date DATE            Wikipedia dump date (e.g. 20230601)
  --language [LANG ...]  Language code (e.g. en). If missing, all languages are processed
  --cache-dir DIR        Cache directory for 🤗 Datasets
  --mirror MIRROR        Mirror URL
```

For instance, for faster downloads of the dumps, use the mirror option:

```sh
python -m prep.build \
    --date 20230601 \
    --language bs \
    --mirror https://mirror.accum.se/mirror/wikimedia.org/dumps/
```

It will download the dumps at around 60MB/s instead of the capped speed
(~4MB/s) from <https://dumps.wikimedia.org>. The script will skip existing
directories, allowing you to run the script in several passes.

Notes:

- These instructions build upon the build process of the
  [Wikipedia](https://huggingface.co/datasets/wikipedia) 🤗 Dataset. HF did a
  fantastic job, I just pushed it a bit further.
- Be aware that not all mirrors contain all dumps. For instance mirror.accum.se
  does not contain dumps for languages such as be-x-old or cbk-zam. My own
  solution is to run a first pass using the aforementioned mirror, and a second
  pass with the official `https://dumps.wikimedia.org` site (omitting the
  `--mirror` parameter).

task_categories:text-generation task_categories:fill-mask task_ids:language-modeling task_ids:masked-language-modeling annotations_creators:no-annotation language_creators:crowdsourced multilinguality:multilingual size_categories:n<1K size_categories:1K<n<10K size_categories:10K<n<100K size_categories:100K<n<1M size_categories:1M<n<10M source_datasets:original language:ab language:ace language:ady language:af language:ak language:als language:alt language:am language:ami language:an language:ang language:anp language:ar language:arc language:ary language:arz language:as language:ast language:atj language:av language:avk language:awa language:ay language:az language:azb language:ba language:ban language:bar language:bcl language:be language:bg language:bh language:bi language:bjn language:blk language:bm language:bn language:bo language:bpy language:br language:bs language:bug language:bxr language:ca language:cdo language:ce language:ceb language:ch language:cho language:chr language:chy language:ckb language:co language:cr language:crh language:cs language:csb language:cu language:cv language:cy language:da language:dag language:de language:din language:diq language:dsb language:dty language:dv language:dz language:ee language:el language:eml language:eo language:es language:et language:eu language:ext language:fa language:fat language:ff language:fi language:fj language:fo language:fr language:frp language:frr language:fur language:fy language:ga language:gag language:gan language:gcr language:gd language:gl language:glk language:gn language:gom language:gor language:got language:gu language:guc language:gur language:guw language:gv language:ha language:hak language:haw language:he language:hi language:hif language:ho language:hr language:hsb language:ht language:hu language:hy language:hyw language:ia language:id language:ie language:ig language:ii language:ik language:ilo language:inh language:io language:is language:it language:iu language:ja language:jam language:jbo language:jv language:ka language:kaa language:kab language:kbd language:kbp language:kcg language:kg language:ki language:kj language:kk language:kl language:km language:kn language:ko language:koi language:krc language:ks language:ksh language:ku language:kv language:kw language:ky language:la language:lad language:lb language:lbe language:lez language:lfn language:lg language:li language:lij language:lld language:lmo language:ln language:lo language:lrc language:lt language:ltg language:lv language:mad language:mai language:mdf language:mg language:mh language:mhr language:mi language:min language:mk language:ml language:mn language:mni language:mnw language:mr language:mrj language:ms language:mt language:mus language:mwl language:my language:myv language:mzn language:nah language:nap language:nds language:ne language:new language:ng language:nia language:nl language:nn language:no language:nov language:nqo language:nrm language:nso language:nv language:ny language:oc language:olo language:om language:or language:os language:pa language:pag language:pam language:pap language:pcd language:pcm language:pdc language:pfl language:pi language:pih language:pl language:pms language:pnb language:pnt language:ps language:pt language:pwn language:qu language:rm language:rmy language:rn language:ro language:ru language:rue language:rw language:sa language:sah language:sat language:sc language:scn language:sco language:sd language:se language:sg language:sh language:shi language:shn language:si language:sk language:skr language:sl language:sm language:smn language:sn language:so language:sq language:sr language:srn language:ss language:st language:stq language:su language:sv language:sw language:szl language:szy language:ta language:tay language:tcy language:te language:tet language:tg language:th language:ti language:tk language:tl language:tn language:to language:tpi language:tr language:trv language:ts language:tt language:tum language:tw language:ty language:tyv language:udm language:ug language:uk language:ur language:uz language:ve language:vec language:vep language:vi language:vls language:vo language:wa language:war language:wo language:wuu language:xal language:xh language:xmf language:yi language:yo language:za language:zea language:zh language:zu license:cc-by-sa-3.0 license:gfdl region:us
{"annotations_creators": ["no-annotation"], "language_creators": ["crowdsourced"], "language": ["ab", "ace", "ady", "af", "ak", "als", "alt", "am", "ami", "an", "ang", "anp", "ar", "arc", "ary", "arz", "as", "ast", "atj", "av", "avk", "awa", "ay", "az", "azb", "ba", "ban", "bar", "bcl", "be", "bg", "bh", "bi", "bjn", "blk", "bm", "bn", "bo", "bpy", "br", "bs", "bug", "bxr", "ca", "cdo", "ce", "ceb", "ch", "cho", "chr", "chy", "ckb", "co", "cr", "crh", "cs", "csb", "cu", "cv", "cy", "da", "dag", "de", "din", "diq", "dsb", "dty", "dv", "dz", "ee", "el", "eml", "eo", "es", "et", "eu", "ext", "fa", "fat", "ff", "fi", "fj", "fo", "fr", "frp", "frr", "fur", "fy", "ga", "gag", "gan", "gcr", "gd", "gl", "glk", "gn", "gom", "gor", "got", "gu", "guc", "gur", "guw", "gv", "ha", "hak", "haw", "he", "hi", "hif", "ho", "hr", "hsb", "ht", "hu", "hy", "hyw", "ia", "id", "ie", "ig", "ii", "ik", "ilo", "inh", "io", "is", "it", "iu", "ja", "jam", "jbo", "jv", "ka", "kaa", "kab", "kbd", "kbp", "kcg", "kg", "ki", "kj", "kk", "kl", "km", "kn", "ko", "koi", "krc", "ks", "ksh", "ku", "kv", "kw", "ky", "la", "lad", "lb", "lbe", "lez", "lfn", "lg", "li", "lij", "lld", "lmo", "ln", "lo", "lrc", "lt", "ltg", "lv", "mad", "mai", "mdf", "mg", "mh", "mhr", "mi", "min", "mk", "ml", "mn", "mni", "mnw", "mr", "mrj", "ms", "mt", "mus", "mwl", "my", "myv", "mzn", "nah", "nap", "nds", "ne", "new", "ng", "nia", "nl", "nn", false, "nov", "nqo", "nrm", "nso", "nv", "ny", "oc", "olo", "om", "or", "os", "pa", "pag", "pam", "pap", "pcd", "pcm", "pdc", "pfl", "pi", "pih", "pl", "pms", "pnb", "pnt", "ps", "pt", "pwn", "qu", "rm", "rmy", "rn", "ro", "ru", "rue", "rw", "sa", "sah", "sat", "sc", "scn", "sco", "sd", "se", "sg", "sh", "shi", "shn", "si", "sk", "skr", "sl", "sm", "smn", "sn", "so", "sq", "sr", "srn", "ss", "st", "stq", "su", "sv", "sw", "szl", "szy", "ta", "tay", "tcy", "te", "tet", "tg", "th", "ti", "tk", "tl", "tn", "to", "tpi", "tr", "trv", "ts", "tt", "tum", "tw", "ty", "tyv", "udm", "ug", "uk", "ur", "uz", "ve", "vec", "vep", "vi", "vls", "vo", "wa", "war", "wo", "wuu", "xal", "xh", "xmf", "yi", "yo", "za", "zea", "zh", "zu"], "license": ["cc-by-sa-3.0", "gfdl"], "multilinguality": ["multilingual"], "size_categories": ["n<1K", "1K<n<10K", "10K<n<100K", "100K<n<1M", "1M<n<10M"], "source_datasets": ["original"], "task_categories": ["text-generation", "fill-mask"], "task_ids": ["language-modeling", "masked-language-modeling"], "pretty_name": "Wikipedia", "language_bcp47": ["bat-smg", "be-x-old", "cbk-zam", "fiu-vro", "map-bms", "nds-nl", "roa-rup", "roa-tara", "simple", "zh-classical", "zh-min-nan", "zh-yue"], "dataset_info": [{"config_name": "20230601.ab", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4183525, "num_examples": 6114}], "download_size": 1172328, "dataset_size": 4183525}, {"config_name": "20230601.ace", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4887561, "num_examples": 12839}], "download_size": 1473823, "dataset_size": 4887561}, {"config_name": "20230601.ady", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 613082, "num_examples": 609}], "download_size": 280249, "dataset_size": 613082}, {"config_name": "20230601.af", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 220678901, "num_examples": 108170}], "download_size": 121238071, "dataset_size": 220678901}, {"config_name": "20230601.ak", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 189, "num_examples": 1}], "download_size": 3045, "dataset_size": 189}, {"config_name": "20230601.als", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 80615079, "num_examples": 29804}], "download_size": 48883379, "dataset_size": 80615079}, {"config_name": "20230601.alt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5786027, "num_examples": 1082}], "download_size": 2401701, "dataset_size": 5786027}, {"config_name": "20230601.am", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 24009050, "num_examples": 13839}], "download_size": 10615909, "dataset_size": 24009050}, {"config_name": "20230601.ami", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3865236, "num_examples": 1570}], "download_size": 2006639, "dataset_size": 3865236}, {"config_name": "20230601.an", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 56295233, "num_examples": 43744}], "download_size": 29055888, "dataset_size": 56295233}, {"config_name": "20230601.ang", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2854073, "num_examples": 4019}], "download_size": 1756372, "dataset_size": 2854073}, {"config_name": "20230601.anp", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9055032, "num_examples": 2736}], "download_size": 3270423, "dataset_size": 9055032}, {"config_name": "20230601.ar", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3052201469, "num_examples": 1205403}], "download_size": 1319905253, "dataset_size": 3052201469}, {"config_name": "20230601.arc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 830073, "num_examples": 1925}], "download_size": 360590, "dataset_size": 830073}, {"config_name": "20230601.ary", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10007364, "num_examples": 6703}], "download_size": 4094420, "dataset_size": 10007364}, {"config_name": "20230601.arz", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1364641408, "num_examples": 1617770}], "download_size": 306336320, "dataset_size": 1364641408}, {"config_name": "20230601.as", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 86645223, "num_examples": 11988}], "download_size": 33149841, "dataset_size": 86645223}, {"config_name": "20230601.ast", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 470349731, "num_examples": 132550}], "download_size": 271011784, "dataset_size": 470349731}, {"config_name": "20230601.atj", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 993287, "num_examples": 1965}], "download_size": 502890, "dataset_size": 993287}, {"config_name": "20230601.av", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5996158, "num_examples": 3392}], "download_size": 2514243, "dataset_size": 5996158}, {"config_name": "20230601.avk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 31189461, "num_examples": 27493}], "download_size": 7729144, "dataset_size": 31189461}, {"config_name": "20230601.awa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3588050, "num_examples": 3701}], "download_size": 1230725, "dataset_size": 3588050}, {"config_name": "20230601.ay", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4357283, "num_examples": 5287}], "download_size": 1736571, "dataset_size": 4357283}, {"config_name": "20230601.az", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 425710145, "num_examples": 194486}], "download_size": 225589717, "dataset_size": 425710145}, {"config_name": "20230601.azb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 186034971, "num_examples": 243041}], "download_size": 46251265, "dataset_size": 186034971}, {"config_name": "20230601.ba", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 293142247, "num_examples": 62907}], "download_size": 120320323, "dataset_size": 293142247}, {"config_name": "20230601.ban", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16509353, "num_examples": 19293}], "download_size": 6302437, "dataset_size": 16509353}, {"config_name": "20230601.bar", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 36001708, "num_examples": 26978}], "download_size": 21611902, "dataset_size": 36001708}, {"config_name": "20230601.bat-smg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 7536614, "num_examples": 17181}], "download_size": 3411835, "dataset_size": 7536614}, {"config_name": "20230601.be-x-old", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 244894736, "num_examples": 82917}], "download_size": 110733701, "dataset_size": 244894736}, {"config_name": "20230601.bcl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 18259970, "num_examples": 13934}], "download_size": 10086356, "dataset_size": 18259970}, {"config_name": "20230601.be", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 606416485, "num_examples": 231617}], "download_size": 280474552, "dataset_size": 606416485}, {"config_name": "20230601.bg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1080390968, "num_examples": 291361}], "download_size": 506945262, "dataset_size": 1080390968}, {"config_name": "20230601.bh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16078510, "num_examples": 8446}], "download_size": 5648960, "dataset_size": 16078510}, {"config_name": "20230601.bi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 398357, "num_examples": 1539}], "download_size": 200277, "dataset_size": 398357}, {"config_name": "20230601.bjn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6755874, "num_examples": 10379}], "download_size": 3265979, "dataset_size": 6755874}, {"config_name": "20230601.blk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 24413622, "num_examples": 2725}], "download_size": 7356285, "dataset_size": 24413622}, {"config_name": "20230601.bm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 473185, "num_examples": 1221}], "download_size": 261438, "dataset_size": 473185}, {"config_name": "20230601.bn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 913676298, "num_examples": 138515}], "download_size": 330147337, "dataset_size": 913676298}, {"config_name": "20230601.bo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 132034426, "num_examples": 12434}], "download_size": 38687191, "dataset_size": 132034426}, {"config_name": "20230601.bpy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 42862119, "num_examples": 25167}], "download_size": 6532133, "dataset_size": 42862119}, {"config_name": "20230601.br", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 84044684, "num_examples": 79959}], "download_size": 48952223, "dataset_size": 84044684}, {"config_name": "20230601.bs", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 190816695, "num_examples": 92065}], "download_size": 106053913, "dataset_size": 190816695}, {"config_name": "20230601.bug", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3433134, "num_examples": 15873}], "download_size": 815878, "dataset_size": 3433134}, {"config_name": "20230601.bxr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6695205, "num_examples": 2791}], "download_size": 3078381, "dataset_size": 6695205}, {"config_name": "20230601.ca", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1918941844, "num_examples": 728483}], "download_size": 1113762234, "dataset_size": 1918941844}, {"config_name": "20230601.cbk-zam", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2808337, "num_examples": 3307}], "download_size": 1261855, "dataset_size": 2808337}, {"config_name": "20230601.cdo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5010639, "num_examples": 16234}], "download_size": 1949302, "dataset_size": 5010639}, {"config_name": "20230601.ce", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 726468413, "num_examples": 599863}], "download_size": 86627608, "dataset_size": 726468413}, {"config_name": "20230601.ceb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4569352784, "num_examples": 6124009}], "download_size": 926156250, "dataset_size": 4569352784}, {"config_name": "20230601.ch", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 187255, "num_examples": 573}], "download_size": 96403, "dataset_size": 187255}, {"config_name": "20230601.cho", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 7974, "num_examples": 14}], "download_size": 9782, "dataset_size": 7974}, {"config_name": "20230601.chr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 764388, "num_examples": 1113}], "download_size": 341232, "dataset_size": 764388}, {"config_name": "20230601.chy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 149009, "num_examples": 801}], "download_size": 76580, "dataset_size": 149009}, {"config_name": "20230601.ckb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 101248717, "num_examples": 49928}], "download_size": 40379289, "dataset_size": 101248717}, {"config_name": "20230601.co", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8069524, "num_examples": 6565}], "download_size": 4650142, "dataset_size": 8069524}, {"config_name": "20230601.cr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 50625, "num_examples": 182}], "download_size": 26509, "dataset_size": 50625}, {"config_name": "20230601.crh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9056373, "num_examples": 25642}], "download_size": 3453399, "dataset_size": 9056373}, {"config_name": "20230601.cs", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1529727976, "num_examples": 525205}], "download_size": 966856046, "dataset_size": 1529727976}, {"config_name": "20230601.csb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3739371, "num_examples": 5478}], "download_size": 2049003, "dataset_size": 3739371}, {"config_name": "20230601.cu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 975765, "num_examples": 1221}], "download_size": 395563, "dataset_size": 975765}, {"config_name": "20230601.cv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 81019358, "num_examples": 51407}], "download_size": 29189010, "dataset_size": 81019358}, {"config_name": "20230601.cy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 304314230, "num_examples": 278927}], "download_size": 111093453, "dataset_size": 304314230}, {"config_name": "20230601.da", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 540186121, "num_examples": 291721}], "download_size": 326825586, "dataset_size": 540186121}, {"config_name": "20230601.dag", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8116697, "num_examples": 8850}], "download_size": 3469680, "dataset_size": 8116697}, {"config_name": "20230601.de", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9446726072, "num_examples": 2801769}], "download_size": 5752429951, "dataset_size": 9446726072}, {"config_name": "20230601.din", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 554422, "num_examples": 506}], "download_size": 334229, "dataset_size": 554422}, {"config_name": "20230601.diq", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 19300910, "num_examples": 40589}], "download_size": 7469118, "dataset_size": 19300910}, {"config_name": "20230601.dsb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3303132, "num_examples": 3357}], "download_size": 1923763, "dataset_size": 3303132}, {"config_name": "20230601.dty", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6972841, "num_examples": 3625}], "download_size": 2497168, "dataset_size": 6972841}, {"config_name": "20230601.dv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13916007, "num_examples": 4344}], "download_size": 5255070, "dataset_size": 13916007}, {"config_name": "20230601.dz", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8517069, "num_examples": 777}], "download_size": 2474869, "dataset_size": 8517069}, {"config_name": "20230601.ee", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 844062, "num_examples": 1164}], "download_size": 464418, "dataset_size": 844062}, {"config_name": "20230601.el", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1314451459, "num_examples": 222598}], "download_size": 627997252, "dataset_size": 1314451459}, {"config_name": "20230601.eml", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3605037, "num_examples": 12945}], "download_size": 1681847, "dataset_size": 3605037}, {"config_name": "20230601.en", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 21325670826, "num_examples": 6660918}], "download_size": 12512970849, "dataset_size": 21325670826}, {"config_name": "20230601.eo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 508055613, "num_examples": 337291}], "download_size": 294377264, "dataset_size": 508055613}, {"config_name": "20230601.es", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5889963046, "num_examples": 1805012}], "download_size": 3477902737, "dataset_size": 5889963046}, {"config_name": "20230601.eu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 547125100, "num_examples": 405840}], "download_size": 264099434, "dataset_size": 547125100}, {"config_name": "20230601.ext", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4182030, "num_examples": 3636}], "download_size": 2631658, "dataset_size": 4182030}, {"config_name": "20230601.fa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1851617207, "num_examples": 964236}], "download_size": 759372155, "dataset_size": 1851617207}, {"config_name": "20230601.fat", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1933259, "num_examples": 1046}], "download_size": 1067434, "dataset_size": 1933259}, {"config_name": "20230601.ff", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1401981, "num_examples": 1484}], "download_size": 824781, "dataset_size": 1401981}, {"config_name": "20230601.fi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1125659121, "num_examples": 553519}], "download_size": 678674705, "dataset_size": 1125659121}, {"config_name": "20230601.fiu-vro", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4773469, "num_examples": 6559}], "download_size": 2464729, "dataset_size": 4773469}, {"config_name": "20230601.fj", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 593373, "num_examples": 1283}], "download_size": 323108, "dataset_size": 593373}, {"config_name": "20230601.fo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 15058635, "num_examples": 13954}], "download_size": 8633381, "dataset_size": 15058635}, {"config_name": "20230601.fr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 7910192478, "num_examples": 2525926}], "download_size": 4618774275, "dataset_size": 7910192478}, {"config_name": "20230601.frp", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3517265, "num_examples": 5689}], "download_size": 1847765, "dataset_size": 3517265}, {"config_name": "20230601.frr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10292357, "num_examples": 17260}], "download_size": 5084999, "dataset_size": 10292357}, {"config_name": "20230601.fur", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4062291, "num_examples": 3967}], "download_size": 2401534, "dataset_size": 4062291}, {"config_name": "20230601.fy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 130189677, "num_examples": 51506}], "download_size": 73624821, "dataset_size": 130189677}, {"config_name": "20230601.ga", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 59266973, "num_examples": 58579}], "download_size": 33377343, "dataset_size": 59266973}, {"config_name": "20230601.gag", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2405210, "num_examples": 2966}], "download_size": 1319553, "dataset_size": 2405210}, {"config_name": "20230601.gan", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2878337, "num_examples": 6691}], "download_size": 1485195, "dataset_size": 2878337}, {"config_name": "20230601.gcr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2335924, "num_examples": 2397}], "download_size": 1344338, "dataset_size": 2335924}, {"config_name": "20230601.gd", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14026914, "num_examples": 16018}], "download_size": 7175920, "dataset_size": 14026914}, {"config_name": "20230601.gl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 483432936, "num_examples": 196473}], "download_size": 287329100, "dataset_size": 483432936}, {"config_name": "20230601.glk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6067898, "num_examples": 7035}], "download_size": 2372761, "dataset_size": 6067898}, {"config_name": "20230601.gn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6754303, "num_examples": 5298}], "download_size": 3702975, "dataset_size": 6754303}, {"config_name": "20230601.gom", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 30830020, "num_examples": 4250}], "download_size": 11258918, "dataset_size": 30830020}, {"config_name": "20230601.gor", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6111487, "num_examples": 14556}], "download_size": 2036928, "dataset_size": 6111487}, {"config_name": "20230601.got", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1518930, "num_examples": 1005}], "download_size": 626840, "dataset_size": 1518930}, {"config_name": "20230601.gu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 120869564, "num_examples": 30357}], "download_size": 39339802, "dataset_size": 120869564}, {"config_name": "20230601.guc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 916033, "num_examples": 578}], "download_size": 547551, "dataset_size": 916033}, {"config_name": "20230601.gur", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1414225, "num_examples": 954}], "download_size": 753483, "dataset_size": 1414225}, {"config_name": "20230601.guw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1894278, "num_examples": 1301}], "download_size": 1027313, "dataset_size": 1894278}, {"config_name": "20230601.gv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5969707, "num_examples": 5954}], "download_size": 3155779, "dataset_size": 5969707}, {"config_name": "20230601.ha", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 62945985, "num_examples": 27905}], "download_size": 35159511, "dataset_size": 62945985}, {"config_name": "20230601.hak", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4493017, "num_examples": 10183}], "download_size": 1875697, "dataset_size": 4493017}, {"config_name": "20230601.haw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1648045, "num_examples": 2580}], "download_size": 681202, "dataset_size": 1648045}, {"config_name": "20230601.he", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1890961532, "num_examples": 325534}], "download_size": 955373507, "dataset_size": 1890961532}, {"config_name": "20230601.hi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 652930384, "num_examples": 160068}], "download_size": 230339569, "dataset_size": 652930384}, {"config_name": "20230601.hif", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5670768, "num_examples": 10975}], "download_size": 2708959, "dataset_size": 5670768}, {"config_name": "20230601.ho", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3450, "num_examples": 3}], "download_size": 7714, "dataset_size": 3450}, {"config_name": "20230601.hsb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 15650862, "num_examples": 13929}], "download_size": 7422054, "dataset_size": 15650862}, {"config_name": "20230601.ht", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 54468681, "num_examples": 69778}], "download_size": 21591458, "dataset_size": 54468681}, {"config_name": "20230601.hu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1490296647, "num_examples": 526030}], "download_size": 904279478, "dataset_size": 1490296647}, {"config_name": "20230601.hy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1142467643, "num_examples": 297933}], "download_size": 477398053, "dataset_size": 1142467643}, {"config_name": "20230601.hyw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 57478946, "num_examples": 10933}], "download_size": 26499417, "dataset_size": 57478946}, {"config_name": "20230601.ia", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16183963, "num_examples": 27939}], "download_size": 8108662, "dataset_size": 16183963}, {"config_name": "20230601.id", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1086885042, "num_examples": 648383}], "download_size": 575124507, "dataset_size": 1086885042}, {"config_name": "20230601.ie", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6482834, "num_examples": 11705}], "download_size": 2881031, "dataset_size": 6482834}, {"config_name": "20230601.ig", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 45043729, "num_examples": 16970}], "download_size": 23565907, "dataset_size": 45043729}, {"config_name": "20230601.ii", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8921, "num_examples": 14}], "download_size": 14936, "dataset_size": 8921}, {"config_name": "20230601.ik", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 190236, "num_examples": 823}], "download_size": 109460, "dataset_size": 190236}, {"config_name": "20230601.ilo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16860855, "num_examples": 15379}], "download_size": 7350161, "dataset_size": 16860855}, {"config_name": "20230601.inh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2697943, "num_examples": 2108}], "download_size": 1257824, "dataset_size": 2697943}, {"config_name": "20230601.io", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 37291268, "num_examples": 38155}], "download_size": 16629067, "dataset_size": 37291268}, {"config_name": "20230601.is", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 86487184, "num_examples": 56795}], "download_size": 51372350, "dataset_size": 86487184}, {"config_name": "20230601.it", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4826403309, "num_examples": 1812514}], "download_size": 2926177870, "dataset_size": 4826403309}, {"config_name": "20230601.iu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 284349, "num_examples": 564}], "download_size": 132368, "dataset_size": 284349}, {"config_name": "20230601.ja", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6913216645, "num_examples": 1373311}], "download_size": 3923535785, "dataset_size": 6913216645}, {"config_name": "20230601.jam", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1140551, "num_examples": 1771}], "download_size": 700995, "dataset_size": 1140551}, {"config_name": "20230601.jbo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2521508, "num_examples": 1390}], "download_size": 888087, "dataset_size": 2521508}, {"config_name": "20230601.jv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 70703094, "num_examples": 73024}], "download_size": 36199167, "dataset_size": 70703094}, {"config_name": "20230601.ka", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 693108151, "num_examples": 168185}], "download_size": 237719175, "dataset_size": 693108151}, {"config_name": "20230601.kaa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4584133, "num_examples": 3560}], "download_size": 2620141, "dataset_size": 4584133}, {"config_name": "20230601.kab", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4374017, "num_examples": 5800}], "download_size": 2570505, "dataset_size": 4374017}, {"config_name": "20230601.kbd", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3034249, "num_examples": 1637}], "download_size": 1317388, "dataset_size": 3034249}, {"config_name": "20230601.kbp", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3571606, "num_examples": 1918}], "download_size": 1794790, "dataset_size": 3571606}, {"config_name": "20230601.kcg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 663326, "num_examples": 825}], "download_size": 350587, "dataset_size": 663326}, {"config_name": "20230601.kg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 463083, "num_examples": 1333}], "download_size": 240321, "dataset_size": 463083}, {"config_name": "20230601.ki", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 453178, "num_examples": 1635}], "download_size": 243544, "dataset_size": 453178}, {"config_name": "20230601.kj", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5190, "num_examples": 5}], "download_size": 10453, "dataset_size": 5190}, {"config_name": "20230601.kk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 488955469, "num_examples": 237304}], "download_size": 176872369, "dataset_size": 488955469}, {"config_name": "20230601.kl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 312839, "num_examples": 298}], "download_size": 193192, "dataset_size": 312839}, {"config_name": "20230601.km", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 102051337, "num_examples": 11784}], "download_size": 35067125, "dataset_size": 102051337}, {"config_name": "20230601.kn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 394061570, "num_examples": 30793}], "download_size": 143867617, "dataset_size": 394061570}, {"config_name": "20230601.ko", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1374136790, "num_examples": 635278}], "download_size": 777760206, "dataset_size": 1374136790}, {"config_name": "20230601.koi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5077608, "num_examples": 3487}], "download_size": 1880469, "dataset_size": 5077608}, {"config_name": "20230601.krc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4592333, "num_examples": 2098}], "download_size": 2019043, "dataset_size": 4592333}, {"config_name": "20230601.ks", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2380920, "num_examples": 4060}], "download_size": 849849, "dataset_size": 2380920}, {"config_name": "20230601.ksh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3110398, "num_examples": 2945}], "download_size": 2004743, "dataset_size": 3110398}, {"config_name": "20230601.ku", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 42327613, "num_examples": 59529}], "download_size": 21970440, "dataset_size": 42327613}, {"config_name": "20230601.kv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9221030, "num_examples": 5589}], "download_size": 3676356, "dataset_size": 9221030}, {"config_name": "20230601.kw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4653320, "num_examples": 7070}], "download_size": 2695687, "dataset_size": 4653320}, {"config_name": "20230601.ky", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 168214006, "num_examples": 80594}], "download_size": 64353836, "dataset_size": 168214006}, {"config_name": "20230601.la", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 139977277, "num_examples": 137851}], "download_size": 75850224, "dataset_size": 139977277}, {"config_name": "20230601.lad", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4820385, "num_examples": 3638}], "download_size": 2703040, "dataset_size": 4820385}, {"config_name": "20230601.lb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 87567860, "num_examples": 61757}], "download_size": 49791518, "dataset_size": 87567860}, {"config_name": "20230601.lbe", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 698292, "num_examples": 1276}], "download_size": 282486, "dataset_size": 698292}, {"config_name": "20230601.lez", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9785097, "num_examples": 4256}], "download_size": 3849506, "dataset_size": 9785097}, {"config_name": "20230601.lfn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8850905, "num_examples": 4805}], "download_size": 5189938, "dataset_size": 8850905}, {"config_name": "20230601.lg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6771716, "num_examples": 4016}], "download_size": 3634293, "dataset_size": 6771716}, {"config_name": "20230601.li", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 29183994, "num_examples": 14308}], "download_size": 17566220, "dataset_size": 29183994}, {"config_name": "20230601.lij", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11088927, "num_examples": 11132}], "download_size": 6042920, "dataset_size": 11088927}, {"config_name": "20230601.lld", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 45325217, "num_examples": 158242}], "download_size": 12436563, "dataset_size": 45325217}, {"config_name": "20230601.lmo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 42267433, "num_examples": 71061}], "download_size": 18724770, "dataset_size": 42267433}, {"config_name": "20230601.ln", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2024697, "num_examples": 3515}], "download_size": 1115171, "dataset_size": 2024697}, {"config_name": "20230601.lo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14729412, "num_examples": 4928}], "download_size": 5382036, "dataset_size": 14729412}, {"config_name": "20230601.lrc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 144, "num_examples": 1}], "download_size": 2723, "dataset_size": 144}, {"config_name": "20230601.lt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 331252602, "num_examples": 208114}], "download_size": 191925990, "dataset_size": 331252602}, {"config_name": "20230601.ltg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 901980, "num_examples": 1044}], "download_size": 522213, "dataset_size": 901980}, {"config_name": "20230601.lv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 220969643, "num_examples": 120295}], "download_size": 126161867, "dataset_size": 220969643}, {"config_name": "20230601.mad", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1325061, "num_examples": 1103}], "download_size": 764579, "dataset_size": 1325061}, {"config_name": "20230601.mai", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 21215977, "num_examples": 14622}], "download_size": 6041134, "dataset_size": 21215977}, {"config_name": "20230601.map-bms", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5400186, "num_examples": 13554}], "download_size": 2420169, "dataset_size": 5400186}, {"config_name": "20230601.mdf", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4033455, "num_examples": 3473}], "download_size": 1513534, "dataset_size": 4033455}, {"config_name": "20230601.mg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 71936817, "num_examples": 95675}], "download_size": 21206762, "dataset_size": 71936817}, {"config_name": "20230601.mh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11524, "num_examples": 8}], "download_size": 16877, "dataset_size": 11524}, {"config_name": "20230601.mhr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 19030836, "num_examples": 11016}], "download_size": 6821706, "dataset_size": 19030836}, {"config_name": "20230601.mi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4120867, "num_examples": 7855}], "download_size": 1016905, "dataset_size": 4120867}, {"config_name": "20230601.min", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 118484114, "num_examples": 226953}], "download_size": 25401691, "dataset_size": 118484114}, {"config_name": "20230601.mk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 633734922, "num_examples": 136723}], "download_size": 263383509, "dataset_size": 633734922}, {"config_name": "20230601.ml", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 485143578, "num_examples": 84794}], "download_size": 179727029, "dataset_size": 485143578}, {"config_name": "20230601.mn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 88813927, "num_examples": 23385}], "download_size": 40026827, "dataset_size": 88813927}, {"config_name": "20230601.mni", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9790220, "num_examples": 10877}], "download_size": 2193774, "dataset_size": 9790220}, {"config_name": "20230601.mnw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 45579901, "num_examples": 3184}], "download_size": 13207357, "dataset_size": 45579901}, {"config_name": "20230601.mr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 254646708, "num_examples": 92898}], "download_size": 79982313, "dataset_size": 254646708}, {"config_name": "20230601.mrj", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8729899, "num_examples": 10542}], "download_size": 3278742, "dataset_size": 8729899}, {"config_name": "20230601.ms", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 410354637, "num_examples": 365491}], "download_size": 206610861, "dataset_size": 410354637}, {"config_name": "20230601.mt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 26613613, "num_examples": 5369}], "download_size": 15563924, "dataset_size": 26613613}, {"config_name": "20230601.mus", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 922, "num_examples": 2}], "download_size": 5286, "dataset_size": 922}, {"config_name": "20230601.mwl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 19284605, "num_examples": 4474}], "download_size": 11469001, "dataset_size": 19284605}, {"config_name": "20230601.my", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 310836677, "num_examples": 108750}], "download_size": 84350660, "dataset_size": 310836677}, {"config_name": "20230601.myv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11073788, "num_examples": 7910}], "download_size": 4560227, "dataset_size": 11073788}, {"config_name": "20230601.mzn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14682517, "num_examples": 15995}], "download_size": 4856126, "dataset_size": 14682517}, {"config_name": "20230601.nah", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2843124, "num_examples": 6654}], "download_size": 1347633, "dataset_size": 2843124}, {"config_name": "20230601.nap", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6365024, "num_examples": 14849}], "download_size": 3169570, "dataset_size": 6365024}, {"config_name": "20230601.nds", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 92743798, "num_examples": 84225}], "download_size": 47925882, "dataset_size": 92743798}, {"config_name": "20230601.nds-nl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13432115, "num_examples": 7669}], "download_size": 8207550, "dataset_size": 13432115}, {"config_name": "20230601.ne", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 105562688, "num_examples": 32084}], "download_size": 36335987, "dataset_size": 105562688}, {"config_name": "20230601.new", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 159067466, "num_examples": 73004}], "download_size": 20472096, "dataset_size": 159067466}, {"config_name": "20230601.ng", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 68090, "num_examples": 21}], "download_size": 52355, "dataset_size": 68090}, {"config_name": "20230601.nia", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1793045, "num_examples": 1638}], "download_size": 908004, "dataset_size": 1793045}, {"config_name": "20230601.nl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2607286503, "num_examples": 2123556}], "download_size": 1451716829, "dataset_size": 2607286503}, {"config_name": "20230601.nn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 233905017, "num_examples": 165610}], "download_size": 132674509, "dataset_size": 233905017}, {"config_name": "20230601.no", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1018553680, "num_examples": 611542}], "download_size": 594771430, "dataset_size": 1018553680}, {"config_name": "20230601.nov", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 912652, "num_examples": 1626}], "download_size": 466451, "dataset_size": 912652}, {"config_name": "20230601.nqo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8295905, "num_examples": 1577}], "download_size": 3503359, "dataset_size": 8295905}, {"config_name": "20230601.nrm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3212495, "num_examples": 4887}], "download_size": 1504411, "dataset_size": 3212495}, {"config_name": "20230601.nso", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2753446, "num_examples": 8617}], "download_size": 912548, "dataset_size": 2753446}, {"config_name": "20230601.nv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16785014, "num_examples": 22189}], "download_size": 3271175, "dataset_size": 16785014}, {"config_name": "20230601.ny", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1693443, "num_examples": 1133}], "download_size": 937213, "dataset_size": 1693443}, {"config_name": "20230601.oc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 117818984, "num_examples": 88886}], "download_size": 62764519, "dataset_size": 117818984}, {"config_name": "20230601.olo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3122448, "num_examples": 4514}], "download_size": 1707016, "dataset_size": 3122448}, {"config_name": "20230601.om", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3057811, "num_examples": 1574}], "download_size": 1720686, "dataset_size": 3057811}, {"config_name": "20230601.or", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 71342568, "num_examples": 16793}], "download_size": 25347488, "dataset_size": 71342568}, {"config_name": "20230601.os", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 12975022, "num_examples": 17066}], "download_size": 5519425, "dataset_size": 12975022}, {"config_name": "20230601.pa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 205173613, "num_examples": 49955}], "download_size": 78370120, "dataset_size": 205173613}, {"config_name": "20230601.pag", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1336264, "num_examples": 2638}], "download_size": 417192, "dataset_size": 1336264}, {"config_name": "20230601.pam", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8241795, "num_examples": 8935}], "download_size": 4231831, "dataset_size": 8241795}, {"config_name": "20230601.pap", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3662048, "num_examples": 3237}], "download_size": 2098802, "dataset_size": 3662048}, {"config_name": "20230601.pcd", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5622299, "num_examples": 5639}], "download_size": 3094652, "dataset_size": 5622299}, {"config_name": "20230601.pcm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1531576, "num_examples": 954}], "download_size": 937573, "dataset_size": 1531576}, {"config_name": "20230601.pdc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1196915, "num_examples": 2162}], "download_size": 688667, "dataset_size": 1196915}, {"config_name": "20230601.pfl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3682829, "num_examples": 2756}], "download_size": 1962515, "dataset_size": 3682829}, {"config_name": "20230601.pi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1134003, "num_examples": 3056}], "download_size": 196632, "dataset_size": 1134003}, {"config_name": "20230601.pih", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 378374, "num_examples": 930}], "download_size": 236668, "dataset_size": 378374}, {"config_name": "20230601.pl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2904184909, "num_examples": 1569515}], "download_size": 1787531053, "dataset_size": 2904184909}, {"config_name": "20230601.pms", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 34301415, "num_examples": 67899}], "download_size": 11986805, "dataset_size": 34301415}, {"config_name": "20230601.pnb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 298316454, "num_examples": 70562}], "download_size": 130650981, "dataset_size": 298316454}, {"config_name": "20230601.pnt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 675000, "num_examples": 535}], "download_size": 298222, "dataset_size": 675000}, {"config_name": "20230601.ps", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 104012780, "num_examples": 19565}], "download_size": 48710783, "dataset_size": 104012780}, {"config_name": "20230601.pt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2693736720, "num_examples": 1103446}], "download_size": 1571347957, "dataset_size": 2693736720}, {"config_name": "20230601.pwn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 800565, "num_examples": 380}], "download_size": 446595, "dataset_size": 800565}, {"config_name": "20230601.qu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16631588, "num_examples": 23909}], "download_size": 7575996, "dataset_size": 16631588}, {"config_name": "20230601.rm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 17822525, "num_examples": 3815}], "download_size": 10339459, "dataset_size": 17822525}, {"config_name": "20230601.rmy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 491195, "num_examples": 930}], "download_size": 285442, "dataset_size": 491195}, {"config_name": "20230601.rn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 522745, "num_examples": 805}], "download_size": 295575, "dataset_size": 522745}, {"config_name": "20230601.ro", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 834681972, "num_examples": 440015}], "download_size": 466488330, "dataset_size": 834681972}, {"config_name": "20230601.roa-rup", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1713384, "num_examples": 1409}], "download_size": 955926, "dataset_size": 1713384}, {"config_name": "20230601.roa-tara", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 7418561, "num_examples": 9337}], "download_size": 3970663, "dataset_size": 7418561}, {"config_name": "20230601.ru", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10097718899, "num_examples": 1918942}], "download_size": 4880008552, "dataset_size": 10097718899}, {"config_name": "20230601.rue", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 12975836, "num_examples": 8703}], "download_size": 6269020, "dataset_size": 12975836}, {"config_name": "20230601.rw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10794817, "num_examples": 7425}], "download_size": 6009979, "dataset_size": 10794817}, {"config_name": "20230601.sa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 69233233, "num_examples": 12101}], "download_size": 23590461, "dataset_size": 69233233}, {"config_name": "20230601.sah", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 47530889, "num_examples": 16598}], "download_size": 21213858, "dataset_size": 47530889}, {"config_name": "20230601.sat", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 35005528, "num_examples": 8264}], "download_size": 12124520, "dataset_size": 35005528}, {"config_name": "20230601.sc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 12683528, "num_examples": 7540}], "download_size": 7650423, "dataset_size": 12683528}, {"config_name": "20230601.scn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 17672274, "num_examples": 26507}], "download_size": 10210177, "dataset_size": 17672274}, {"config_name": "20230601.sco", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 43796852, "num_examples": 36206}], "download_size": 24764727, "dataset_size": 43796852}, {"config_name": "20230601.sd", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 36672141, "num_examples": 16882}], "download_size": 17409382, "dataset_size": 36672141}, {"config_name": "20230601.se", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3600247, "num_examples": 8040}], "download_size": 1814982, "dataset_size": 3600247}, {"config_name": "20230601.sg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 127791, "num_examples": 548}], "download_size": 63800, "dataset_size": 127791}, {"config_name": "20230601.sh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 569915575, "num_examples": 458272}], "download_size": 270502498, "dataset_size": 569915575}, {"config_name": "20230601.shi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2195129, "num_examples": 1544}], "download_size": 1311300, "dataset_size": 2195129}, {"config_name": "20230601.shn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 33233508, "num_examples": 13706}], "download_size": 8107005, "dataset_size": 33233508}, {"config_name": "20230601.si", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 135560965, "num_examples": 22574}], "download_size": 52870973, "dataset_size": 135560965}, {"config_name": "20230601.sk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 410287543, "num_examples": 240597}], "download_size": 237984111, "dataset_size": 410287543}, {"config_name": "20230601.skr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 22294235, "num_examples": 5739}], "download_size": 9744982, "dataset_size": 22294235}, {"config_name": "20230601.sl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 444732062, "num_examples": 181212}], "download_size": 263697513, "dataset_size": 444732062}, {"config_name": "20230601.sm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 891597, "num_examples": 1143}], "download_size": 485815, "dataset_size": 891597}, {"config_name": "20230601.smn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5526668, "num_examples": 5094}], "download_size": 2710998, "dataset_size": 5526668}, {"config_name": "20230601.sn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9252554, "num_examples": 10917}], "download_size": 4738498, "dataset_size": 9252554}, {"config_name": "20230601.so", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14893759, "num_examples": 10812}], "download_size": 8617659, "dataset_size": 14893759}, {"config_name": "20230601.sq", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 197206847, "num_examples": 100423}], "download_size": 110414776, "dataset_size": 197206847}, {"config_name": "20230601.sr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1690745100, "num_examples": 671352}], "download_size": 695586988, "dataset_size": 1690745100}, {"config_name": "20230601.srn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 649044, "num_examples": 1218}], "download_size": 214987, "dataset_size": 649044}, {"config_name": "20230601.ss", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 861417, "num_examples": 720}], "download_size": 489383, "dataset_size": 861417}, {"config_name": "20230601.st", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 934954, "num_examples": 1073}], "download_size": 517491, "dataset_size": 934954}, {"config_name": "20230601.stq", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4929355, "num_examples": 4129}], "download_size": 2878034, "dataset_size": 4929355}, {"config_name": "20230601.su", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 47909002, "num_examples": 61490}], "download_size": 19683635, "dataset_size": 47909002}, {"config_name": "20230601.sv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2133848723, "num_examples": 2564263}], "download_size": 1002020509, "dataset_size": 2133848723}, {"config_name": "20230601.sw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 71857907, "num_examples": 77334}], "download_size": 35252918, "dataset_size": 71857907}, {"config_name": "20230601.szl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 21335080, "num_examples": 56652}], "download_size": 7284436, "dataset_size": 21335080}, {"config_name": "20230601.szy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10412319, "num_examples": 4709}], "download_size": 5572825, "dataset_size": 10412319}, {"config_name": "20230601.tay", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2779734, "num_examples": 2595}], "download_size": 1147869, "dataset_size": 2779734}, {"config_name": "20230601.tcy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11968976, "num_examples": 2173}], "download_size": 4524692, "dataset_size": 11968976}, {"config_name": "20230601.te", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 705766405, "num_examples": 83107}], "download_size": 206360536, "dataset_size": 705766405}, {"config_name": "20230601.tet", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1457614, "num_examples": 1460}], "download_size": 739227, "dataset_size": 1457614}, {"config_name": "20230601.tg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 145506377, "num_examples": 109839}], "download_size": 48637192, "dataset_size": 145506377}, {"config_name": "20230601.th", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 987873133, "num_examples": 156445}], "download_size": 365894157, "dataset_size": 987873133}, {"config_name": "20230601.ti", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 665363, "num_examples": 433}], "download_size": 328037, "dataset_size": 665363}, {"config_name": "20230601.tk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 12580480, "num_examples": 7836}], "download_size": 6951103, "dataset_size": 12580480}, {"config_name": "20230601.tl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 82731267, "num_examples": 44797}], "download_size": 44058126, "dataset_size": 82731267}, {"config_name": "20230601.tn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3562981, "num_examples": 1162}], "download_size": 1244173, "dataset_size": 3562981}, {"config_name": "20230601.to", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1074947, "num_examples": 1848}], "download_size": 510687, "dataset_size": 1074947}, {"config_name": "20230601.tpi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 450891, "num_examples": 1390}], "download_size": 236441, "dataset_size": 450891}, {"config_name": "20230601.tr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 965186144, "num_examples": 524184}], "download_size": 543958666, "dataset_size": 965186144}, {"config_name": "20230601.trv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4873244, "num_examples": 1809}], "download_size": 2635461, "dataset_size": 4873244}, {"config_name": "20230601.ts", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 841497, "num_examples": 769}], "download_size": 451958, "dataset_size": 841497}, {"config_name": "20230601.tt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 679276199, "num_examples": 500608}], "download_size": 128386602, "dataset_size": 679276199}, {"config_name": "20230601.tum", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8395079, "num_examples": 14169}], "download_size": 3225881, "dataset_size": 8395079}, {"config_name": "20230601.tw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6562128, "num_examples": 3608}], "download_size": 3389042, "dataset_size": 6562128}, {"config_name": "20230601.ty", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 324678, "num_examples": 1348}], "download_size": 145184, "dataset_size": 324678}, {"config_name": "20230601.tyv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14032235, "num_examples": 3459}], "download_size": 6378954, "dataset_size": 14032235}, {"config_name": "20230601.udm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6918258, "num_examples": 5586}], "download_size": 2937644, "dataset_size": 6918258}, {"config_name": "20230601.ug", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 41939834, "num_examples": 8557}], "download_size": 17588763, "dataset_size": 41939834}, {"config_name": "20230601.uk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4815765166, "num_examples": 1266287}], "download_size": 2257591520, "dataset_size": 4815765166}, {"config_name": "20230601.ur", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 394375073, "num_examples": 194435}], "download_size": 160552761, "dataset_size": 394375073}, {"config_name": "20230601.uz", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 372775375, "num_examples": 241353}], "download_size": 196367714, "dataset_size": 372775375}, {"config_name": "20230601.ve", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 347015, "num_examples": 836}], "download_size": 159547, "dataset_size": 347015}, {"config_name": "20230601.vec", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 37671800, "num_examples": 69181}], "download_size": 16029908, "dataset_size": 37671800}, {"config_name": "20230601.vep", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11259222, "num_examples": 6851}], "download_size": 6196150, "dataset_size": 11259222}, {"config_name": "20230601.vi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1584847634, "num_examples": 1283785}], "download_size": 731354374, "dataset_size": 1584847634}, {"config_name": "20230601.vls", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11296047, "num_examples": 7824}], "download_size": 6952370, "dataset_size": 11296047}, {"config_name": "20230601.vo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 18943004, "num_examples": 33641}], "download_size": 6379410, "dataset_size": 18943004}, {"config_name": "20230601.wa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11990482, "num_examples": 11858}], "download_size": 7144929, "dataset_size": 11990482}, {"config_name": "20230601.war", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 468715357, "num_examples": 1266238}], "download_size": 109807953, "dataset_size": 468715357}, {"config_name": "20230601.wo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3498671, "num_examples": 1719}], "download_size": 2076485, "dataset_size": 3498671}, {"config_name": "20230601.wuu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 24986530, "num_examples": 42950}], "download_size": 15960262, "dataset_size": 24986530}, {"config_name": "20230601.xal", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1386014, "num_examples": 2307}], "download_size": 508481, "dataset_size": 1386014}, {"config_name": "20230601.xh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2320277, "num_examples": 1601}], "download_size": 1444732, "dataset_size": 2320277}, {"config_name": "20230601.xmf", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 36557690, "num_examples": 17705}], "download_size": 12535173, "dataset_size": 36557690}, {"config_name": "20230601.yi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 36031133, "num_examples": 15297}], "download_size": 16153644, "dataset_size": 36031133}, {"config_name": "20230601.yo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 18018480, "num_examples": 33179}], "download_size": 8274108, "dataset_size": 18018480}, {"config_name": "20230601.za", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1276590, "num_examples": 2722}], "download_size": 642448, "dataset_size": 1276590}, {"config_name": "20230601.zea", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5059421, "num_examples": 5756}], "download_size": 2547904, "dataset_size": 5059421}, {"config_name": "20230601.zh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2720688196, "num_examples": 1357881}], "download_size": 1718953037, "dataset_size": 2720688196}, {"config_name": "20230601.zh-classical", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14617535, "num_examples": 12513}], "download_size": 9882532, "dataset_size": 14617535}, {"config_name": "20230601.zh-min-nan", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 159218053, "num_examples": 432531}], "download_size": 37371610, "dataset_size": 159218053}, {"config_name": "20230601.zh-yue", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 107325669, "num_examples": 131542}], "download_size": 63294114, "dataset_size": 107325669}, {"config_name": "20230601.zu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6915666, "num_examples": 11381}], "download_size": 3683813, "dataset_size": 6915666}, {"config_name": "20230601.hr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 438311404, "num_examples": 200747}], "download_size": 275098294, "dataset_size": 438311404}, {"config_name": "20230601.simple", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 282844880, "num_examples": 231233}], "download_size": 154520600, "dataset_size": 282844880}, {"config_name": "20230601.ta", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 789472198, "num_examples": 156273}], "download_size": 258263767, "dataset_size": 789472198}, {"config_name": "20230901.ab", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4257828, "num_examples": 6135}], "download_size": 1204070, "dataset_size": 4257828}, {"config_name": "20230901.ace", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4988748, "num_examples": 12932}], "download_size": 1532859, "dataset_size": 4988748}, {"config_name": "20230901.ady", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 732900, "num_examples": 656}], "download_size": 334202, "dataset_size": 732900}, {"config_name": "20230901.af", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 223836122, "num_examples": 110683}], "download_size": 122868601, "dataset_size": 223836122}, {"config_name": "20230901.ak", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 189, "num_examples": 1}], "download_size": 3045, "dataset_size": 189}, {"config_name": "20230901.als", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 81066470, "num_examples": 29914}], "download_size": 49151942, "dataset_size": 81066470}, {"config_name": "20230901.alt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6370197, "num_examples": 1076}], "download_size": 2683190, "dataset_size": 6370197}, {"config_name": "20230901.am", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 24108874, "num_examples": 13863}], "download_size": 10659605, "dataset_size": 24108874}, {"config_name": "20230901.ami", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4376488, "num_examples": 1613}], "download_size": 2207864, "dataset_size": 4376488}, {"config_name": "20230901.an", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 57157273, "num_examples": 44090}], "download_size": 29392661, "dataset_size": 57157273}, {"config_name": "20230901.ang", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2899899, "num_examples": 4106}], "download_size": 1782699, "dataset_size": 2899899}, {"config_name": "20230901.anp", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9238243, "num_examples": 2753}], "download_size": 3338080, "dataset_size": 9238243}, {"config_name": "20230901.ar", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3090850739, "num_examples": 1214692}], "download_size": 1336764394, "dataset_size": 3090850739}, {"config_name": "20230901.arc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 837851, "num_examples": 1935}], "download_size": 364313, "dataset_size": 837851}, {"config_name": "20230901.ary", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10716445, "num_examples": 7181}], "download_size": 4413789, "dataset_size": 10716445}, {"config_name": "20230901.arz", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1371439747, "num_examples": 1619204}], "download_size": 309552126, "dataset_size": 1371439747}, {"config_name": "20230901.as", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 88616101, "num_examples": 12209}], "download_size": 33925273, "dataset_size": 88616101}, {"config_name": "20230901.ast", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 470680707, "num_examples": 133219}], "download_size": 271143532, "dataset_size": 470680707}, {"config_name": "20230901.atj", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1009452, "num_examples": 1967}], "download_size": 512377, "dataset_size": 1009452}, {"config_name": "20230901.av", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6136668, "num_examples": 3420}], "download_size": 2568423, "dataset_size": 6136668}, {"config_name": "20230901.avk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 31833142, "num_examples": 28141}], "download_size": 7911635, "dataset_size": 31833142}, {"config_name": "20230901.awa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3591539, "num_examples": 3696}], "download_size": 1233124, "dataset_size": 3591539}, {"config_name": "20230901.ay", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4378141, "num_examples": 5348}], "download_size": 1748641, "dataset_size": 4378141}, {"config_name": "20230901.az", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 430470815, "num_examples": 195659}], "download_size": 228140471, "dataset_size": 430470815}, {"config_name": "20230901.azb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 186776266, "num_examples": 243263}], "download_size": 46619566, "dataset_size": 186776266}, {"config_name": "20230901.ba", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 296321332, "num_examples": 63134}], "download_size": 121809783, "dataset_size": 296321332}, {"config_name": "20230901.ban", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 17383384, "num_examples": 20242}], "download_size": 6524686, "dataset_size": 17383384}, {"config_name": "20230901.bar", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 36251706, "num_examples": 27040}], "download_size": 21762636, "dataset_size": 36251706}, {"config_name": "20230901.bat-smg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 7584027, "num_examples": 17214}], "download_size": 3437198, "dataset_size": 7584027}, {"config_name": "20230901.be-x-old", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 249911330, "num_examples": 83778}], "download_size": 113105161, "dataset_size": 249911330}, {"config_name": "20230901.bcl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 19285430, "num_examples": 14723}], "download_size": 10682007, "dataset_size": 19285430}, {"config_name": "20230901.be", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 618711883, "num_examples": 234760}], "download_size": 286395236, "dataset_size": 618711883}, {"config_name": "20230901.bg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1095408838, "num_examples": 293306}], "download_size": 514238024, "dataset_size": 1095408838}, {"config_name": "20230901.bh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16433197, "num_examples": 8552}], "download_size": 5775459, "dataset_size": 16433197}, {"config_name": "20230901.bi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 405238, "num_examples": 1544}], "download_size": 204286, "dataset_size": 405238}, {"config_name": "20230901.bjn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6761698, "num_examples": 10460}], "download_size": 3255595, "dataset_size": 6761698}, {"config_name": "20230901.blk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 25837114, "num_examples": 2923}], "download_size": 7802724, "dataset_size": 25837114}, {"config_name": "20230901.bm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 591154, "num_examples": 1254}], "download_size": 324954, "dataset_size": 591154}, {"config_name": "20230901.bn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 945095157, "num_examples": 141288}], "download_size": 340510394, "dataset_size": 945095157}, {"config_name": "20230901.bo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 132468794, "num_examples": 12826}], "download_size": 38750901, "dataset_size": 132468794}, {"config_name": "20230901.bpy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 42975074, "num_examples": 25165}], "download_size": 6557544, "dataset_size": 42975074}, {"config_name": "20230901.br", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 84959382, "num_examples": 83342}], "download_size": 49373423, "dataset_size": 84959382}, {"config_name": "20230901.bs", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 192322421, "num_examples": 92325}], "download_size": 106973603, "dataset_size": 192322421}, {"config_name": "20230901.bug", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3433942, "num_examples": 15877}], "download_size": 816476, "dataset_size": 3433942}, {"config_name": "20230901.bxr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6686504, "num_examples": 2791}], "download_size": 3073419, "dataset_size": 6686504}, {"config_name": "20230901.ca", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1942397691, "num_examples": 733807}], "download_size": 1127952357, "dataset_size": 1942397691}, {"config_name": "20230901.cbk-zam", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1997943, "num_examples": 3276}], "download_size": 776590, "dataset_size": 1997943}, {"config_name": "20230901.cdo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5085776, "num_examples": 16406}], "download_size": 1972779, "dataset_size": 5085776}, {"config_name": "20230901.ce", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 729121943, "num_examples": 600961}], "download_size": 87442481, "dataset_size": 729121943}, {"config_name": "20230901.ceb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4568428530, "num_examples": 6122999}], "download_size": 925715583, "dataset_size": 4568428530}, {"config_name": "20230901.ch", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 187141, "num_examples": 591}], "download_size": 93248, "dataset_size": 187141}, {"config_name": "20230901.cho", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 7974, "num_examples": 14}], "download_size": 9782, "dataset_size": 7974}, {"config_name": "20230901.chr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 768617, "num_examples": 1121}], "download_size": 343463, "dataset_size": 768617}, {"config_name": "20230901.chy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 145752, "num_examples": 800}], "download_size": 74383, "dataset_size": 145752}, {"config_name": "20230901.ckb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 105393226, "num_examples": 51534}], "download_size": 42196297, "dataset_size": 105393226}, {"config_name": "20230901.co", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9828777, "num_examples": 7286}], "download_size": 5312668, "dataset_size": 9828777}, {"config_name": "20230901.cr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 54526, "num_examples": 176}], "download_size": 34910, "dataset_size": 54526}, {"config_name": "20230901.crh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9450530, "num_examples": 26893}], "download_size": 3578677, "dataset_size": 9450530}, {"config_name": "20230901.cs", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1552256812, "num_examples": 531017}], "download_size": 981191812, "dataset_size": 1552256812}, {"config_name": "20230901.csb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3748403, "num_examples": 5480}], "download_size": 2055688, "dataset_size": 3748403}, {"config_name": "20230901.cu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 981478, "num_examples": 1237}], "download_size": 397764, "dataset_size": 981478}, {"config_name": "20230901.cv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 81463626, "num_examples": 51647}], "download_size": 29416321, "dataset_size": 81463626}, {"config_name": "20230901.cy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 305551170, "num_examples": 279341}], "download_size": 111947867, "dataset_size": 305551170}, {"config_name": "20230901.da", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 544417184, "num_examples": 294196}], "download_size": 329369262, "dataset_size": 544417184}, {"config_name": "20230901.dag", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11405576, "num_examples": 9584}], "download_size": 4905465, "dataset_size": 11405576}, {"config_name": "20230901.de", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9552907552, "num_examples": 2828561}], "download_size": 5816126238, "dataset_size": 9552907552}, {"config_name": "20230901.din", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 562639, "num_examples": 511}], "download_size": 339141, "dataset_size": 562639}, {"config_name": "20230901.diq", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 19574906, "num_examples": 41541}], "download_size": 7581584, "dataset_size": 19574906}, {"config_name": "20230901.dsb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3314217, "num_examples": 3376}], "download_size": 1930644, "dataset_size": 3314217}, {"config_name": "20230901.dty", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6999985, "num_examples": 3629}], "download_size": 2505457, "dataset_size": 6999985}, {"config_name": "20230901.dv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13919491, "num_examples": 4345}], "download_size": 5255676, "dataset_size": 13919491}, {"config_name": "20230901.dz", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8837256, "num_examples": 787}], "download_size": 2571127, "dataset_size": 8837256}, {"config_name": "20230901.ee", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 881798, "num_examples": 1172}], "download_size": 482924, "dataset_size": 881798}, {"config_name": "20230901.el", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1335513979, "num_examples": 225623}], "download_size": 637838917, "dataset_size": 1335513979}, {"config_name": "20230901.eml", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3620183, "num_examples": 12954}], "download_size": 1687294, "dataset_size": 3620183}, {"config_name": "20230901.en", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 21550145456, "num_examples": 6705754}], "download_size": 12639246876, "dataset_size": 21550145456}, {"config_name": "20230901.eo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 517650573, "num_examples": 342419}], "download_size": 299082818, "dataset_size": 517650573}, {"config_name": "20230901.es", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5977729133, "num_examples": 1826609}], "download_size": 3528834297, "dataset_size": 5977729133}, {"config_name": "20230901.et", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 436983600, "num_examples": 239195}], "download_size": 266302500, "dataset_size": 436983600}, {"config_name": "20230901.eu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 555867111, "num_examples": 408841}], "download_size": 269449522, "dataset_size": 555867111}, {"config_name": "20230901.ext", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4334809, "num_examples": 3737}], "download_size": 2724237, "dataset_size": 4334809}, {"config_name": "20230901.fa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1879857088, "num_examples": 972647}], "download_size": 771735257, "dataset_size": 1879857088}, {"config_name": "20230901.fat", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2016722, "num_examples": 1113}], "download_size": 1115327, "dataset_size": 2016722}, {"config_name": "20230901.ff", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1619659, "num_examples": 1929}], "download_size": 951246, "dataset_size": 1619659}, {"config_name": "20230901.fi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1138299674, "num_examples": 558359}], "download_size": 686112933, "dataset_size": 1138299674}, {"config_name": "20230901.fiu-vro", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4789834, "num_examples": 6572}], "download_size": 2475758, "dataset_size": 4789834}, {"config_name": "20230901.fj", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 600984, "num_examples": 1291}], "download_size": 325888, "dataset_size": 600984}, {"config_name": "20230901.fo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 15387671, "num_examples": 14054}], "download_size": 8835604, "dataset_size": 15387671}, {"config_name": "20230901.fr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8004882292, "num_examples": 2549364}], "download_size": 4674130728, "dataset_size": 8004882292}, {"config_name": "20230901.frp", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3646051, "num_examples": 5744}], "download_size": 1899883, "dataset_size": 3646051}, {"config_name": "20230901.frr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10513932, "num_examples": 17708}], "download_size": 5190719, "dataset_size": 10513932}, {"config_name": "20230901.fur", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4073954, "num_examples": 3977}], "download_size": 2408634, "dataset_size": 4073954}, {"config_name": "20230901.fy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 133127089, "num_examples": 52120}], "download_size": 75305215, "dataset_size": 133127089}, {"config_name": "20230901.ga", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 60113068, "num_examples": 58940}], "download_size": 33805587, "dataset_size": 60113068}, {"config_name": "20230901.gag", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2405444, "num_examples": 2967}], "download_size": 1319216, "dataset_size": 2405444}, {"config_name": "20230901.gan", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2905828, "num_examples": 6739}], "download_size": 1504592, "dataset_size": 2905828}, {"config_name": "20230901.gcr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2338042, "num_examples": 2398}], "download_size": 1345374, "dataset_size": 2338042}, {"config_name": "20230901.gd", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14057133, "num_examples": 16034}], "download_size": 7199577, "dataset_size": 14057133}, {"config_name": "20230901.gl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 489325069, "num_examples": 198354}], "download_size": 291176228, "dataset_size": 489325069}, {"config_name": "20230901.glk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6078167, "num_examples": 7046}], "download_size": 2379845, "dataset_size": 6078167}, {"config_name": "20230901.gn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6869059, "num_examples": 5475}], "download_size": 3777263, "dataset_size": 6869059}, {"config_name": "20230901.gom", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 30886509, "num_examples": 4257}], "download_size": 11274837, "dataset_size": 30886509}, {"config_name": "20230901.gor", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6131050, "num_examples": 14572}], "download_size": 2047896, "dataset_size": 6131050}, {"config_name": "20230901.got", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1533270, "num_examples": 1012}], "download_size": 633392, "dataset_size": 1533270}, {"config_name": "20230901.gu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 121284600, "num_examples": 30413}], "download_size": 39504567, "dataset_size": 121284600}, {"config_name": "20230901.guc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 939870, "num_examples": 618}], "download_size": 556772, "dataset_size": 939870}, {"config_name": "20230901.gur", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1620565, "num_examples": 1119}], "download_size": 820347, "dataset_size": 1620565}, {"config_name": "20230901.guw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1900240, "num_examples": 1303}], "download_size": 1030888, "dataset_size": 1900240}, {"config_name": "20230901.gv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6030196, "num_examples": 6009}], "download_size": 3195985, "dataset_size": 6030196}, {"config_name": "20230901.ha", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 73654886, "num_examples": 33752}], "download_size": 40714314, "dataset_size": 73654886}, {"config_name": "20230901.hak", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4509695, "num_examples": 10238}], "download_size": 1879146, "dataset_size": 4509695}, {"config_name": "20230901.haw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1672431, "num_examples": 2615}], "download_size": 694045, "dataset_size": 1672431}, {"config_name": "20230901.he", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1927823110, "num_examples": 330733}], "download_size": 974031783, "dataset_size": 1927823110}, {"config_name": "20230901.hi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 667221249, "num_examples": 162285}], "download_size": 235641052, "dataset_size": 667221249}, {"config_name": "20230901.hif", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5676100, "num_examples": 10981}], "download_size": 2709810, "dataset_size": 5676100}, {"config_name": "20230901.ho", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3450, "num_examples": 3}], "download_size": 7714, "dataset_size": 3450}, {"config_name": "20230901.hr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 441122356, "num_examples": 201819}], "download_size": 276842760, "dataset_size": 441122356}, {"config_name": "20230901.hsb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 15657332, "num_examples": 13949}], "download_size": 7427955, "dataset_size": 15657332}, {"config_name": "20230901.ht", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 54641623, "num_examples": 70002}], "download_size": 21699003, "dataset_size": 54641623}, {"config_name": "20230901.hu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1505652559, "num_examples": 529609}], "download_size": 913575039, "dataset_size": 1505652559}, {"config_name": "20230901.hy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1167174995, "num_examples": 301853}], "download_size": 488665605, "dataset_size": 1167174995}, {"config_name": "20230901.hyw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 59286603, "num_examples": 11644}], "download_size": 27305593, "dataset_size": 59286603}, {"config_name": "20230901.ia", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16319168, "num_examples": 28081}], "download_size": 8200366, "dataset_size": 16319168}, {"config_name": "20230901.id", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1110116852, "num_examples": 657990}], "download_size": 587862344, "dataset_size": 1110116852}, {"config_name": "20230901.ie", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6658278, "num_examples": 11811}], "download_size": 2978290, "dataset_size": 6658278}, {"config_name": "20230901.ig", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 55435770, "num_examples": 19892}], "download_size": 28977840, "dataset_size": 55435770}, {"config_name": "20230901.ii", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8921, "num_examples": 14}], "download_size": 14936, "dataset_size": 8921}, {"config_name": "20230901.ik", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 192007, "num_examples": 831}], "download_size": 110667, "dataset_size": 192007}, {"config_name": "20230901.ilo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16853115, "num_examples": 15369}], "download_size": 7345494, "dataset_size": 16853115}, {"config_name": "20230901.inh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2722201, "num_examples": 2121}], "download_size": 1273603, "dataset_size": 2722201}, {"config_name": "20230901.io", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 37616691, "num_examples": 38645}], "download_size": 16826496, "dataset_size": 37616691}, {"config_name": "20230901.is", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 87138239, "num_examples": 57147}], "download_size": 51826151, "dataset_size": 87138239}, {"config_name": "20230901.it", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4879369360, "num_examples": 1824508}], "download_size": 2957576589, "dataset_size": 4879369360}, {"config_name": "20230901.iu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 289114, "num_examples": 561}], "download_size": 136067, "dataset_size": 289114}, {"config_name": "20230901.ja", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6988535462, "num_examples": 1383531}], "download_size": 3966219907, "dataset_size": 6988535462}, {"config_name": "20230901.jam", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1142809, "num_examples": 1775}], "download_size": 702478, "dataset_size": 1142809}, {"config_name": "20230901.jbo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2522674, "num_examples": 1391}], "download_size": 888919, "dataset_size": 2522674}, {"config_name": "20230901.jv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 71017946, "num_examples": 73150}], "download_size": 36394809, "dataset_size": 71017946}, {"config_name": "20230901.ka", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 696934958, "num_examples": 169131}], "download_size": 238964498, "dataset_size": 696934958}, {"config_name": "20230901.kaa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4754449, "num_examples": 3856}], "download_size": 2682618, "dataset_size": 4754449}, {"config_name": "20230901.kab", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4388232, "num_examples": 5825}], "download_size": 2578056, "dataset_size": 4388232}, {"config_name": "20230901.kbd", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3040422, "num_examples": 1656}], "download_size": 1319464, "dataset_size": 3040422}, {"config_name": "20230901.kbp", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3579071, "num_examples": 1922}], "download_size": 1795549, "dataset_size": 3579071}, {"config_name": "20230901.kcg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 728303, "num_examples": 913}], "download_size": 382843, "dataset_size": 728303}, {"config_name": "20230901.kg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 386320, "num_examples": 1325}], "download_size": 206106, "dataset_size": 386320}, {"config_name": "20230901.ki", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 731003, "num_examples": 1647}], "download_size": 408805, "dataset_size": 731003}, {"config_name": "20230901.kj", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5190, "num_examples": 5}], "download_size": 10453, "dataset_size": 5190}, {"config_name": "20230901.kk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 494357868, "num_examples": 237902}], "download_size": 179217175, "dataset_size": 494357868}, {"config_name": "20230901.kl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 313121, "num_examples": 298}], "download_size": 193507, "dataset_size": 313121}, {"config_name": "20230901.km", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 102576754, "num_examples": 11874}], "download_size": 35281246, "dataset_size": 102576754}, {"config_name": "20230901.kn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 399521127, "num_examples": 31136}], "download_size": 145847507, "dataset_size": 399521127}, {"config_name": "20230901.ko", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1401002436, "num_examples": 643723}], "download_size": 792232087, "dataset_size": 1401002436}, {"config_name": "20230901.koi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5102564, "num_examples": 3504}], "download_size": 1887860, "dataset_size": 5102564}, {"config_name": "20230901.krc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4586443, "num_examples": 2098}], "download_size": 2015581, "dataset_size": 4586443}, {"config_name": "20230901.ks", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2828813, "num_examples": 4278}], "download_size": 1074931, "dataset_size": 2828813}, {"config_name": "20230901.ksh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3115805, "num_examples": 2944}], "download_size": 2007139, "dataset_size": 3115805}, {"config_name": "20230901.ku", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 43200623, "num_examples": 59822}], "download_size": 22481749, "dataset_size": 43200623}, {"config_name": "20230901.kv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9244682, "num_examples": 5603}], "download_size": 3687481, "dataset_size": 9244682}, {"config_name": "20230901.kw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4675299, "num_examples": 7088}], "download_size": 2703089, "dataset_size": 4675299}, {"config_name": "20230901.ky", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 168378862, "num_examples": 80665}], "download_size": 64423485, "dataset_size": 168378862}, {"config_name": "20230901.la", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 140689294, "num_examples": 138140}], "download_size": 76340691, "dataset_size": 140689294}, {"config_name": "20230901.lad", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4878588, "num_examples": 3648}], "download_size": 2737222, "dataset_size": 4878588}, {"config_name": "20230901.lb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 88394374, "num_examples": 62131}], "download_size": 50250905, "dataset_size": 88394374}, {"config_name": "20230901.lbe", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 744689, "num_examples": 1277}], "download_size": 304111, "dataset_size": 744689}, {"config_name": "20230901.lez", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9793873, "num_examples": 4264}], "download_size": 3852020, "dataset_size": 9793873}, {"config_name": "20230901.lfn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8912633, "num_examples": 4819}], "download_size": 5206921, "dataset_size": 8912633}, {"config_name": "20230901.lg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6887606, "num_examples": 4041}], "download_size": 3703329, "dataset_size": 6887606}, {"config_name": "20230901.li", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 29373978, "num_examples": 14526}], "download_size": 17641752, "dataset_size": 29373978}, {"config_name": "20230901.lij", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11336209, "num_examples": 11184}], "download_size": 6176932, "dataset_size": 11336209}, {"config_name": "20230901.lld", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 50110703, "num_examples": 180580}], "download_size": 13839995, "dataset_size": 50110703}, {"config_name": "20230901.lmo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 43217251, "num_examples": 72899}], "download_size": 19041052, "dataset_size": 43217251}, {"config_name": "20230901.ln", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2024359, "num_examples": 3531}], "download_size": 1116032, "dataset_size": 2024359}, {"config_name": "20230901.lo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 15117598, "num_examples": 4995}], "download_size": 5527479, "dataset_size": 15117598}, {"config_name": "20230901.lrc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 144, "num_examples": 1}], "download_size": 2723, "dataset_size": 144}, {"config_name": "20230901.lt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 334697442, "num_examples": 210202}], "download_size": 193837594, "dataset_size": 334697442}, {"config_name": "20230901.ltg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 915321, "num_examples": 1070}], "download_size": 530333, "dataset_size": 915321}, {"config_name": "20230901.lv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 224476781, "num_examples": 122266}], "download_size": 128157342, "dataset_size": 224476781}, {"config_name": "20230901.mad", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1504064, "num_examples": 1160}], "download_size": 856724, "dataset_size": 1504064}, {"config_name": "20230901.mai", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 21426268, "num_examples": 14673}], "download_size": 6117668, "dataset_size": 21426268}, {"config_name": "20230901.map-bms", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5413521, "num_examples": 13574}], "download_size": 2427039, "dataset_size": 5413521}, {"config_name": "20230901.mdf", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4558408, "num_examples": 4073}], "download_size": 1688901, "dataset_size": 4558408}, {"config_name": "20230901.mg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 72920973, "num_examples": 96060}], "download_size": 21675187, "dataset_size": 72920973}, {"config_name": "20230901.mh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11524, "num_examples": 8}], "download_size": 16877, "dataset_size": 11524}, {"config_name": "20230901.mhr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 19188080, "num_examples": 11246}], "download_size": 6867184, "dataset_size": 19188080}, {"config_name": "20230901.mi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4159228, "num_examples": 7898}], "download_size": 1039215, "dataset_size": 4159228}, {"config_name": "20230901.min", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 118651753, "num_examples": 227024}], "download_size": 25511300, "dataset_size": 118651753}, {"config_name": "20230901.mk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 640596981, "num_examples": 138453}], "download_size": 266334099, "dataset_size": 640596981}, {"config_name": "20230901.ml", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 490833742, "num_examples": 85451}], "download_size": 181789443, "dataset_size": 490833742}, {"config_name": "20230901.mn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 90537032, "num_examples": 23797}], "download_size": 40809884, "dataset_size": 90537032}, {"config_name": "20230901.mni", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9818372, "num_examples": 10892}], "download_size": 2207828, "dataset_size": 9818372}, {"config_name": "20230901.mnw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 46788079, "num_examples": 3249}], "download_size": 13588244, "dataset_size": 46788079}, {"config_name": "20230901.mr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 260342611, "num_examples": 93653}], "download_size": 81397471, "dataset_size": 260342611}, {"config_name": "20230901.mrj", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8731508, "num_examples": 10542}], "download_size": 3279598, "dataset_size": 8731508}, {"config_name": "20230901.ms", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 419678289, "num_examples": 367463}], "download_size": 211505058, "dataset_size": 419678289}, {"config_name": "20230901.mt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 30536771, "num_examples": 5598}], "download_size": 17850471, "dataset_size": 30536771}, {"config_name": "20230901.mus", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 922, "num_examples": 2}], "download_size": 5286, "dataset_size": 922}, {"config_name": "20230901.mwl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 19321295, "num_examples": 4485}], "download_size": 11488668, "dataset_size": 19321295}, {"config_name": "20230901.my", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 312482214, "num_examples": 109166}], "download_size": 84914025, "dataset_size": 312482214}, {"config_name": "20230901.myv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11131103, "num_examples": 7947}], "download_size": 4586300, "dataset_size": 11131103}, {"config_name": "20230901.mzn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 15830260, "num_examples": 17696}], "download_size": 5258917, "dataset_size": 15830260}, {"config_name": "20230901.nah", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2494573, "num_examples": 6180}], "download_size": 1188515, "dataset_size": 2494573}, {"config_name": "20230901.nap", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6377175, "num_examples": 14868}], "download_size": 3176787, "dataset_size": 6377175}, {"config_name": "20230901.nds", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 92854034, "num_examples": 84258}], "download_size": 48004103, "dataset_size": 92854034}, {"config_name": "20230901.nds-nl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13560241, "num_examples": 7707}], "download_size": 8287716, "dataset_size": 13560241}, {"config_name": "20230901.ne", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 106930147, "num_examples": 32423}], "download_size": 36867790, "dataset_size": 106930147}, {"config_name": "20230901.new", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 159078463, "num_examples": 73003}], "download_size": 20468180, "dataset_size": 159078463}, {"config_name": "20230901.ng", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 68090, "num_examples": 21}], "download_size": 52355, "dataset_size": 68090}, {"config_name": "20230901.nia", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1909528, "num_examples": 1651}], "download_size": 970289, "dataset_size": 1909528}, {"config_name": "20230901.nl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2631597985, "num_examples": 2130944}], "download_size": 1467451759, "dataset_size": 2631597985}, {"config_name": "20230901.nn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 236262183, "num_examples": 166642}], "download_size": 134021748, "dataset_size": 236262183}, {"config_name": "20230901.no", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1027035487, "num_examples": 615107}], "download_size": 599774543, "dataset_size": 1027035487}, {"config_name": "20230901.nov", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 917413, "num_examples": 1636}], "download_size": 469305, "dataset_size": 917413}, {"config_name": "20230901.nqo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8219209, "num_examples": 1571}], "download_size": 3478458, "dataset_size": 8219209}, {"config_name": "20230901.nrm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3215096, "num_examples": 4899}], "download_size": 1505717, "dataset_size": 3215096}, {"config_name": "20230901.nso", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2789807, "num_examples": 8643}], "download_size": 932635, "dataset_size": 2789807}, {"config_name": "20230901.nv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16886983, "num_examples": 22324}], "download_size": 3288156, "dataset_size": 16886983}, {"config_name": "20230901.ny", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1695102, "num_examples": 1133}], "download_size": 938716, "dataset_size": 1695102}, {"config_name": "20230901.oc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 119055715, "num_examples": 89270}], "download_size": 63403412, "dataset_size": 119055715}, {"config_name": "20230901.olo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3152274, "num_examples": 4595}], "download_size": 1716616, "dataset_size": 3152274}, {"config_name": "20230901.om", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3430032, "num_examples": 1911}], "download_size": 1900253, "dataset_size": 3430032}, {"config_name": "20230901.or", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 72723705, "num_examples": 17166}], "download_size": 25879025, "dataset_size": 72723705}, {"config_name": "20230901.os", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13112794, "num_examples": 17446}], "download_size": 5554157, "dataset_size": 13112794}, {"config_name": "20230901.pa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 211148791, "num_examples": 51013}], "download_size": 80668229, "dataset_size": 211148791}, {"config_name": "20230901.pag", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1384685, "num_examples": 2662}], "download_size": 451639, "dataset_size": 1384685}, {"config_name": "20230901.pam", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 8237319, "num_examples": 8951}], "download_size": 4235968, "dataset_size": 8237319}, {"config_name": "20230901.pap", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4105109, "num_examples": 3427}], "download_size": 2353692, "dataset_size": 4105109}, {"config_name": "20230901.pcd", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5680386, "num_examples": 5692}], "download_size": 3127716, "dataset_size": 5680386}, {"config_name": "20230901.pcm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1807444, "num_examples": 1069}], "download_size": 1111719, "dataset_size": 1807444}, {"config_name": "20230901.pdc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1223268, "num_examples": 2182}], "download_size": 696649, "dataset_size": 1223268}, {"config_name": "20230901.pfl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3688761, "num_examples": 2759}], "download_size": 1963616, "dataset_size": 3688761}, {"config_name": "20230901.pi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1133972, "num_examples": 3056}], "download_size": 196617, "dataset_size": 1133972}, {"config_name": "20230901.pih", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 381602, "num_examples": 933}], "download_size": 238696, "dataset_size": 381602}, {"config_name": "20230901.pl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2929578273, "num_examples": 1579326}], "download_size": 1803033674, "dataset_size": 2929578273}, {"config_name": "20230901.pms", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 34318527, "num_examples": 67935}], "download_size": 11997737, "dataset_size": 34318527}, {"config_name": "20230901.pnb", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 303876889, "num_examples": 72240}], "download_size": 133093182, "dataset_size": 303876889}, {"config_name": "20230901.pnt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 630714, "num_examples": 533}], "download_size": 275657, "dataset_size": 630714}, {"config_name": "20230901.ps", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 109664877, "num_examples": 20166}], "download_size": 51380951, "dataset_size": 109664877}, {"config_name": "20230901.pt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2731435653, "num_examples": 1107946}], "download_size": 1593477871, "dataset_size": 2731435653}, {"config_name": "20230901.pwn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 792234, "num_examples": 394}], "download_size": 433617, "dataset_size": 792234}, {"config_name": "20230901.qu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 16754330, "num_examples": 24096}], "download_size": 7651901, "dataset_size": 16754330}, {"config_name": "20230901.rm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 18052223, "num_examples": 3821}], "download_size": 10475947, "dataset_size": 18052223}, {"config_name": "20230901.rmy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 555208, "num_examples": 969}], "download_size": 324565, "dataset_size": 555208}, {"config_name": "20230901.rn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 522604, "num_examples": 808}], "download_size": 295315, "dataset_size": 522604}, {"config_name": "20230901.ro", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 842490285, "num_examples": 441538}], "download_size": 471249050, "dataset_size": 842490285}, {"config_name": "20230901.roa-rup", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1691177, "num_examples": 1409}], "download_size": 953023, "dataset_size": 1691177}, {"config_name": "20230901.roa-tara", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 7435543, "num_examples": 9341}], "download_size": 3982748, "dataset_size": 7435543}, {"config_name": "20230901.ru", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10213314874, "num_examples": 1935562}], "download_size": 4935575161, "dataset_size": 10213314874}, {"config_name": "20230901.rue", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13110982, "num_examples": 8749}], "download_size": 6335689, "dataset_size": 13110982}, {"config_name": "20230901.rw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11946518, "num_examples": 8044}], "download_size": 6640582, "dataset_size": 11946518}, {"config_name": "20230901.sa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 69665685, "num_examples": 12143}], "download_size": 23750145, "dataset_size": 69665685}, {"config_name": "20230901.sah", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 47816835, "num_examples": 16867}], "download_size": 21350955, "dataset_size": 47816835}, {"config_name": "20230901.sat", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 40858282, "num_examples": 9029}], "download_size": 13950418, "dataset_size": 40858282}, {"config_name": "20230901.sc", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 12732368, "num_examples": 7559}], "download_size": 7682010, "dataset_size": 12732368}, {"config_name": "20230901.scn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 17667128, "num_examples": 26519}], "download_size": 10212874, "dataset_size": 17667128}, {"config_name": "20230901.sco", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 43780491, "num_examples": 36169}], "download_size": 24761453, "dataset_size": 43780491}, {"config_name": "20230901.sd", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 36726435, "num_examples": 16894}], "download_size": 17439666, "dataset_size": 36726435}, {"config_name": "20230901.se", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3600162, "num_examples": 8042}], "download_size": 1814812, "dataset_size": 3600162}, {"config_name": "20230901.sg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 130365, "num_examples": 553}], "download_size": 65750, "dataset_size": 130365}, {"config_name": "20230901.sh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 569747500, "num_examples": 458212}], "download_size": 270404350, "dataset_size": 569747500}, {"config_name": "20230901.shi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2348743, "num_examples": 1771}], "download_size": 1347026, "dataset_size": 2348743}, {"config_name": "20230901.shn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 33479127, "num_examples": 13878}], "download_size": 8148046, "dataset_size": 33479127}, {"config_name": "20230901.si", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 136810596, "num_examples": 22893}], "download_size": 53392258, "dataset_size": 136810596}, {"config_name": "20230901.simple", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 287855540, "num_examples": 238150}], "download_size": 157248327, "dataset_size": 287855540}, {"config_name": "20230901.sk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 414483614, "num_examples": 241614}], "download_size": 240700453, "dataset_size": 414483614}, {"config_name": "20230901.skr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 22524450, "num_examples": 5768}], "download_size": 9854778, "dataset_size": 22524450}, {"config_name": "20230901.sl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 451888560, "num_examples": 182364}], "download_size": 268258798, "dataset_size": 451888560}, {"config_name": "20230901.sm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 904339, "num_examples": 1149}], "download_size": 493408, "dataset_size": 904339}, {"config_name": "20230901.smn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5673858, "num_examples": 5333}], "download_size": 2767537, "dataset_size": 5673858}, {"config_name": "20230901.sn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 9587086, "num_examples": 11354}], "download_size": 4889856, "dataset_size": 9587086}, {"config_name": "20230901.so", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13594918, "num_examples": 9003}], "download_size": 7886560, "dataset_size": 13594918}, {"config_name": "20230901.sq", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 204838795, "num_examples": 103850}], "download_size": 114648801, "dataset_size": 204838795}, {"config_name": "20230901.sr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1709332753, "num_examples": 673516}], "download_size": 704099906, "dataset_size": 1709332753}, {"config_name": "20230901.srn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 649208, "num_examples": 1219}], "download_size": 215087, "dataset_size": 649208}, {"config_name": "20230901.ss", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1024219, "num_examples": 890}], "download_size": 574998, "dataset_size": 1024219}, {"config_name": "20230901.st", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 956079, "num_examples": 1094}], "download_size": 523485, "dataset_size": 956079}, {"config_name": "20230901.stq", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4934155, "num_examples": 4132}], "download_size": 2880185, "dataset_size": 4934155}, {"config_name": "20230901.su", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 48039769, "num_examples": 61557}], "download_size": 19764523, "dataset_size": 48039769}, {"config_name": "20230901.sv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2146681766, "num_examples": 2570535}], "download_size": 1009875904, "dataset_size": 2146681766}, {"config_name": "20230901.sw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 72884231, "num_examples": 78444}], "download_size": 35798700, "dataset_size": 72884231}, {"config_name": "20230901.szl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 21412618, "num_examples": 56961}], "download_size": 7330797, "dataset_size": 21412618}, {"config_name": "20230901.szy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 10793237, "num_examples": 4794}], "download_size": 5811192, "dataset_size": 10793237}, {"config_name": "20230901.ta", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 801530157, "num_examples": 158664}], "download_size": 262319221, "dataset_size": 801530157}, {"config_name": "20230901.tay", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2909279, "num_examples": 2715}], "download_size": 1203598, "dataset_size": 2909279}, {"config_name": "20230901.tcy", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 12142146, "num_examples": 2195}], "download_size": 4589253, "dataset_size": 12142146}, {"config_name": "20230901.te", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 719651788, "num_examples": 85840}], "download_size": 211297920, "dataset_size": 719651788}, {"config_name": "20230901.tet", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1464393, "num_examples": 1465}], "download_size": 743636, "dataset_size": 1464393}, {"config_name": "20230901.tg", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 147555847, "num_examples": 110263}], "download_size": 49551755, "dataset_size": 147555847}, {"config_name": "20230901.th", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1002621820, "num_examples": 158289}], "download_size": 371401101, "dataset_size": 1002621820}, {"config_name": "20230901.ti", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 639136, "num_examples": 430}], "download_size": 317759, "dataset_size": 639136}, {"config_name": "20230901.tk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13169481, "num_examples": 7898}], "download_size": 7284367, "dataset_size": 13169481}, {"config_name": "20230901.tl", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 84784414, "num_examples": 45155}], "download_size": 45203377, "dataset_size": 84784414}, {"config_name": "20230901.tn", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3561901, "num_examples": 1160}], "download_size": 1245027, "dataset_size": 3561901}, {"config_name": "20230901.to", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1082372, "num_examples": 1866}], "download_size": 515293, "dataset_size": 1082372}, {"config_name": "20230901.tpi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 457865, "num_examples": 1396}], "download_size": 231303, "dataset_size": 457865}, {"config_name": "20230901.tr", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 984939694, "num_examples": 530830}], "download_size": 554907604, "dataset_size": 984939694}, {"config_name": "20230901.trv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4906787, "num_examples": 1835}], "download_size": 2654525, "dataset_size": 4906787}, {"config_name": "20230901.ts", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 845256, "num_examples": 778}], "download_size": 454559, "dataset_size": 845256}, {"config_name": "20230901.tt", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 680656530, "num_examples": 501002}], "download_size": 129123758, "dataset_size": 680656530}, {"config_name": "20230901.tum", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 13199654, "num_examples": 18591}], "download_size": 5352424, "dataset_size": 13199654}, {"config_name": "20230901.tw", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 7386605, "num_examples": 3717}], "download_size": 3815538, "dataset_size": 7386605}, {"config_name": "20230901.ty", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 333733, "num_examples": 1355}], "download_size": 149306, "dataset_size": 333733}, {"config_name": "20230901.tyv", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14319641, "num_examples": 3481}], "download_size": 6513101, "dataset_size": 14319641}, {"config_name": "20230901.udm", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6975919, "num_examples": 5665}], "download_size": 2952228, "dataset_size": 6975919}, {"config_name": "20230901.ug", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 42219904, "num_examples": 8621}], "download_size": 17716007, "dataset_size": 42219904}, {"config_name": "20230901.uk", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 4910916097, "num_examples": 1285004}], "download_size": 2303106335, "dataset_size": 4910916097}, {"config_name": "20230901.ur", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 402322741, "num_examples": 197343}], "download_size": 164074548, "dataset_size": 402322741}, {"config_name": "20230901.uz", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 385386661, "num_examples": 242726}], "download_size": 203362895, "dataset_size": 385386661}, {"config_name": "20230901.ve", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 349857, "num_examples": 840}], "download_size": 161562, "dataset_size": 349857}, {"config_name": "20230901.vec", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 37883286, "num_examples": 69250}], "download_size": 16164035, "dataset_size": 37883286}, {"config_name": "20230901.vep", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11487509, "num_examples": 6918}], "download_size": 6327017, "dataset_size": 11487509}, {"config_name": "20230901.vi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1606980713, "num_examples": 1287263}], "download_size": 742700712, "dataset_size": 1606980713}, {"config_name": "20230901.vls", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 11310015, "num_examples": 7839}], "download_size": 6960289, "dataset_size": 11310015}, {"config_name": "20230901.vo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 19274897, "num_examples": 34504}], "download_size": 6491359, "dataset_size": 19274897}, {"config_name": "20230901.wa", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 12140372, "num_examples": 11955}], "download_size": 7231141, "dataset_size": 12140372}, {"config_name": "20230901.war", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 467623925, "num_examples": 1266345}], "download_size": 109503863, "dataset_size": 467623925}, {"config_name": "20230901.wo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 3498562, "num_examples": 1718}], "download_size": 2077375, "dataset_size": 3498562}, {"config_name": "20230901.wuu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 25005942, "num_examples": 42969}], "download_size": 15994961, "dataset_size": 25005942}, {"config_name": "20230901.xal", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1390063, "num_examples": 2290}], "download_size": 507117, "dataset_size": 1390063}, {"config_name": "20230901.xh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2415590, "num_examples": 1667}], "download_size": 1503917, "dataset_size": 2415590}, {"config_name": "20230901.xmf", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 37262425, "num_examples": 17949}], "download_size": 12771047, "dataset_size": 37262425}, {"config_name": "20230901.yi", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 36150608, "num_examples": 15329}], "download_size": 16208341, "dataset_size": 36150608}, {"config_name": "20230901.yo", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 18460117, "num_examples": 33495}], "download_size": 8504564, "dataset_size": 18460117}, {"config_name": "20230901.za", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 1359106, "num_examples": 2971}], "download_size": 662982, "dataset_size": 1359106}, {"config_name": "20230901.zea", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5106625, "num_examples": 5834}], "download_size": 2567716, "dataset_size": 5106625}, {"config_name": "20230901.zh", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 2766648619, "num_examples": 1375017}], "download_size": 1748154636, "dataset_size": 2766648619}, {"config_name": "20230901.zh-classical", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 14819164, "num_examples": 12615}], "download_size": 10031693, "dataset_size": 14819164}, {"config_name": "20230901.zh-min-nan", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 159385896, "num_examples": 432644}], "download_size": 37476665, "dataset_size": 159385896}, {"config_name": "20230901.zh-yue", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 108979942, "num_examples": 133155}], "download_size": 64318527, "dataset_size": 108979942}, {"config_name": "20230901.zu", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 6925330, "num_examples": 11486}], "download_size": 3690925, "dataset_size": 6925330}, {"config_name": "20230601.et", "features": [{"name": "id", "dtype": "string"}, {"name": "url", "dtype": "string"}, {"name": "title", "dtype": "string"}, {"name": "text", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 431680309, "num_examples": 236848}], "download_size": 262989758, "dataset_size": 431680309}]}
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-crowdsourced #multilinguality-multilingual #size_categories-n<1K #size_categories-1K<n<10K #size_categories-10K<n<100K #size_categories-100K<n<1M #size_categories-1M<n<10M #source_datasets-original #language-Abkhazian #language-Achinese #language-Adyghe #language-Afrikaans #language-Akan #language-Tosk Albanian #language-Southern Altai #language-Amharic #language-Amis #language-Aragonese #language-Old English (ca. 450-1100) #language-Angika #language-Arabic #language-Official Aramaic (700-300 BCE) #language-Moroccan Arabic #language-Egyptian Arabic #language-Assamese #language-Asturian #language-Atikamekw #language-Avaric #language-Kotava #language-Awadhi #language-Aymara #language-Azerbaijani #language-South Azerbaijani #language-Bashkir #language-Balinese #language-Bavarian #language-Central Bikol #language-Belarusian #language-Bulgarian #language-bh #language-Bislama #language-Banjar #language-Pa'o Karen #language-Bambara #language-Bengali #language-Tibetan #language-Bishnupriya #language-Breton #language-Bosnian #language-Buginese #language-Russia Buriat #language-Catalan #language-Min Dong Chinese #language-Chechen #language-Cebuano #language-Chamorro #language-Choctaw #language-Cherokee #language-Cheyenne #language-Central Kurdish #language-Corsican #language-Cree #language-Crimean Tatar #language-Czech #language-Kashubian #language-Church Slavic #language-Chuvash #language-Welsh #language-Danish #language-Dagbani #language-German #language-Dinka #language-Dimli (individual language) #language-Lower Sorbian #language-Dotyali #language-Dhivehi #language-Dzongkha #language-Ewe #language-Modern Greek (1453-) #language-Emiliano-Romagnolo #language-Esperanto #language-Spanish #language-Estonian #language-Basque #language-Extremaduran #language-Persian #language-Fanti #language-Fulah #language-Finnish #language-Fijian #language-Faroese #language-French #language-Arpitan #language-Northern Frisian #language-Friulian #language-Western Frisian #language-Irish #language-Gagauz #language-Gan Chinese #language-Guianese Creole French #language-Scottish Gaelic #language-Galician #language-Gilaki #language-Guarani #language-Goan Konkani #language-Gorontalo #language-Gothic #language-Gujarati #language-Wayuu #language-Farefare #language-Gun #language-Manx #language-Hausa #language-Hakka Chinese #language-Hawaiian #language-Hebrew #language-Hindi #language-Fiji Hindi #language-Hiri Motu #language-Croatian #language-Upper Sorbian #language-Haitian #language-Hungarian #language-Armenian #language-Western Armenian #language-Interlingua (International Auxiliary Language Association) #language-Indonesian #language-Interlingue #language-Igbo #language-Sichuan Yi #language-Inupiaq #language-Iloko #language-Ingush #language-Ido #language-Icelandic #language-Italian #language-Inuktitut #language-Japanese #language-Jamaican Creole English #language-Lojban #language-Javanese #language-Georgian #language-Kara-Kalpak #language-Kabyle #language-Kabardian #language-Kabiyè #language-Tyap #language-Kongo #language-Kikuyu #language-Kuanyama #language-Kazakh #language-Kalaallisut #language-Khmer #language-Kannada #language-Korean #language-Komi-Permyak #language-Karachay-Balkar #language-Kashmiri #language-Kölsch #language-Kurdish #language-Komi #language-Cornish #language-Kirghiz #language-Latin #language-Ladino #language-Luxembourgish #language-Lak #language-Lezghian #language-Lingua Franca Nova #language-Ganda #language-Limburgan #language-Ligurian #language-Ladin #language-Lombard #language-Lingala #language-Lao #language-Northern Luri #language-Lithuanian #language-Latgalian #language-Latvian #language-Madurese #language-Maithili #language-Moksha #language-Malagasy #language-Marshallese #language-Eastern Mari #language-Maori #language-Minangkabau #language-Macedonian #language-Malayalam #language-Mongolian #language-Manipuri #language-Mon #language-Marathi #language-Western Mari #language-Malay (macrolanguage) #language-Maltese #language-Creek #language-Mirandese #language-Burmese #language-Erzya #language-Mazanderani #language-nah #language-Neapolitan #language-Low German #language-Nepali (macrolanguage) #language-Newari #language-Ndonga #language-Nias #language-Dutch #language-Norwegian Nynorsk #language-Norwegian #language-Novial #language-N'Ko #language-Narom #language-Pedi #language-Navajo #language-Nyanja #language-Occitan (post 1500) #language-Livvi #language-Oromo #language-Oriya (macrolanguage) #language-Ossetian #language-Panjabi #language-Pangasinan #language-Pampanga #language-Papiamento #language-Picard #language-Nigerian Pidgin #language-Pennsylvania German #language-Pfaelzisch #language-Pali #language-Pitcairn-Norfolk #language-Polish #language-Piemontese #language-Western Panjabi #language-Pontic #language-Pushto #language-Portuguese #language-Paiwan #language-Quechua #language-Romansh #language-Vlax Romani #language-Rundi #language-Romanian #language-Russian #language-Rusyn #language-Kinyarwanda #language-Sanskrit #language-Yakut #language-Santali #language-Sardinian #language-Sicilian #language-Scots #language-Sindhi #language-Northern Sami #language-Sango #language-Serbo-Croatian #language-Tachelhit #language-Shan #language-Sinhala #language-Slovak #language-Saraiki #language-Slovenian #language-Samoan #language-Inari Sami #language-Shona #language-Somali #language-Albanian #language-Serbian #language-Sranan Tongo #language-Swati #language-Southern Sotho #language-Saterfriesisch #language-Sundanese #language-Swedish #language-Swahili (macrolanguage) #language-Silesian #language-Sakizaya #language-Tamil #language-Atayal #language-Tulu #language-Telugu #language-Tetum #language-Tajik #language-Thai #language-Tigrinya #language-Turkmen #language-Tagalog #language-Tswana #language-Tonga (Tonga Islands) #language-Tok Pisin #language-Turkish #language-Sediq #language-Tsonga #language-Tatar #language-Tumbuka #language-Twi #language-Tahitian #language-Tuvinian #language-Udmurt #language-Uighur #language-Ukrainian #language-Urdu #language-Uzbek #language-Venda #language-Venetian #language-Veps #language-Vietnamese #language-Vlaams #language-Volapük #language-Walloon #language-Waray (Philippines) #language-Wolof #language-Wu Chinese #language-Kalmyk #language-Xhosa #language-Mingrelian #language-Yiddish #language-Yoruba #language-Zhuang #language-Zeeuws #language-Chinese #language-Zulu #license-cc-by-sa-3.0 #license-gfdl #region-us 

Wikipedia
=========


This Wikipedia dataset contains all available languages for recent dumps. It is
a refresh of the 20220301 wikipedia from
Huggingface, so it has the same license and dataset card details. The benefits
of this dataset are:


* more recent dumps (see table below)
* a few additional languages
* all available languages are preprocessed (including the largests: 'en' and
'ceb')



Source: List of Wikimedia
Languages. A few (9)
Wikimedias are closed, meaning they won't have new pages, but the dumps are
still available. In addition, very few (4) Wikimedias are closed and don't
have dumps anymore.


Release Notes
-------------


'1.2.0'


* chore: Update to 20230901


'1.1.0'


* feat: Add missing estonian (my bad), thanks Chris Ha
* fix: update category lists for az, ceb, ch, hr, ii, lrc, ta, which means
they were all processed again.


'1.0.0'


* chore: File layout is now 'data/{dump}/{lang}/{URL,\*.parquet}'.
Sorry for the radical update, probably won't happen again.
* chore: Parquet files are now sharded (size < 200 MB), allowing parallel
downloads and processing.
* fix: All languages were all processed again because of a bug in the media
and category names, leading to some links not being extracted.
* feat: Add 'en' and 'ceb' which were too big for my Beam DirectRunner at
the time.


Usage
-----




---


Build instructions
------------------


Developer only. This dataset was preprocessed with a Beam DirectRunner as
follows.


### 1. Determine the date of the dump you are interested in


Choose one wikipedia dump, for instance <URL
and identify the date.


### 2. [Optional] Get a refreshed list of languages


This is optional because it not very likely that a new language will have
suddenly appeared since the last version *and* have a significant dataset.


Navigate to <URL and copy the
languages column from the "Detailed list" table (near the end of the page).


Copy that content in the form of a Python list into 'lang\_def.py' (at the top
of the repo) under a new date.


### 3. [Optional] Create Media and Category aliases


In order to properly extract links to images and media in all languages, we
must refresh the two corresponding files. To do so, from the root of the repo,
run


This will create or update these two files at the root of the repo:


* 'media\_aliases.py'
* 'category\_aliases.py'


These files are used in the final step


### 4. Build and prepare the datasets into sharded parquet files


Running this script downloads the wikipedia dumps for each language in
'lang\_def.py' and shards each language dataset into the appropriate number of
shards (max size ~ 250MB).


There are other options:


For instance, for faster downloads of the dumps, use the mirror option:


It will download the dumps at around 60MB/s instead of the capped speed
(~4MB/s) from . The script will skip existing
directories, allowing you to run the script in several passes.


Notes:


* These instructions build upon the build process of the
Wikipedia Dataset. HF did a
fantastic job, I just pushed it a bit further.
* Be aware that not all mirrors contain all dumps. For instance URL
does not contain dumps for languages such as be-x-old or cbk-zam. My own
solution is to run a first pass using the aforementioned mirror, and a second
pass with the official 'URL' site (omitting the
'--mirror' parameter).



### 1. Determine the date of the dump you are interested in


Choose one wikipedia dump, for instance <URL
and identify the date. ### 2. [Optional] Get a refreshed list of languages


This is optional because it not very likely that a new language will have
suddenly appeared since the last version *and* have a significant dataset.


Navigate to <URL and copy the
languages column from the "Detailed list" table (near the end of the page).


Copy that content in the form of a Python list into 'lang\_def.py' (at the top
of the repo) under a new date. ### 3. [Optional] Create Media and Category aliases


In order to properly extract links to images and media in all languages, we
must refresh the two corresponding files. To do so, from the root of the repo,
run


This will create or update these two files at the root of the repo:


* 'media\_aliases.py'
* 'category\_aliases.py'


These files are used in the final step ### 4. Build and prepare the datasets into sharded parquet files


Running this script downloads the wikipedia dumps for each language in
'lang\_def.py' and shards each language dataset into the appropriate number of
shards (max size ~ 250MB).


There are other options:


For instance, for faster downloads of the dumps, use the mirror option:


It will download the dumps at around 60MB/s instead of the capped speed
(~4MB/s) from . The script will skip existing
directories, allowing you to run the script in several passes.


Notes:


* These instructions build upon the build process of the
Wikipedia Dataset. HF did a
fantastic job, I just pushed it a bit further.
* Be aware that not all mirrors contain all dumps. For instance URL
does not contain dumps for languages such as be-x-old or cbk-zam. My own
solution is to run a first pass using the aforementioned mirror, and a second
pass with the official 'URL' site (omitting the
'--mirror' parameter).
TAGS
#task_categories-text-generation #task_categories-fill-mask #task_ids-language-modeling #task_ids-masked-language-modeling #annotations_creators-no-annotation #language_creators-crowdsourced #multilinguality-multilingual #size_categories-n<1K #size_categories-1K<n<10K #size_categories-10K<n<100K #size_categories-100K<n<1M #size_categories-1M<n<10M #source_datasets-original #language-Abkhazian #language-Achinese #language-Adyghe #language-Afrikaans #language-Akan #language-Tosk Albanian #language-Southern Altai #language-Amharic #language-Amis #language-Aragonese #language-Old English (ca. 450-1100) #language-Angika #language-Arabic #language-Official Aramaic (700-300 BCE) #language-Moroccan Arabic #language-Egyptian Arabic #language-Assamese #language-Asturian #language-Atikamekw #language-Avaric #language-Kotava #language-Awadhi #language-Aymara #language-Azerbaijani #language-South Azerbaijani #language-Bashkir #language-Balinese #language-Bavarian #language-Central Bikol #language-Belarusian #language-Bulgarian #language-bh #language-Bislama #language-Banjar #language-Pa'o Karen #language-Bambara #language-Bengali #language-Tibetan #language-Bishnupriya #language-Breton #language-Bosnian #language-Buginese #language-Russia Buriat #language-Catalan #language-Min Dong Chinese #language-Chechen #language-Cebuano #language-Chamorro #language-Choctaw #language-Cherokee #language-Cheyenne #language-Central Kurdish #language-Corsican #language-Cree #language-Crimean Tatar #language-Czech #language-Kashubian #language-Church Slavic #language-Chuvash #language-Welsh #language-Danish #language-Dagbani #language-German #language-Dinka #language-Dimli (individual language) #language-Lower Sorbian #language-Dotyali #language-Dhivehi #language-Dzongkha #language-Ewe #language-Modern Greek (1453-) #language-Emiliano-Romagnolo #language-Esperanto #language-Spanish #language-Estonian #language-Basque #language-Extremaduran #language-Persian #language-Fanti #language-Fulah #language-Finnish #language-Fijian #language-Faroese #language-French #language-Arpitan #language-Northern Frisian #language-Friulian #language-Western Frisian #language-Irish #language-Gagauz #language-Gan Chinese #language-Guianese Creole French #language-Scottish Gaelic #language-Galician #language-Gilaki #language-Guarani #language-Goan Konkani #language-Gorontalo #language-Gothic #language-Gujarati #language-Wayuu #language-Farefare #language-Gun #language-Manx #language-Hausa #language-Hakka Chinese #language-Hawaiian #language-Hebrew #language-Hindi #language-Fiji Hindi #language-Hiri Motu #language-Croatian #language-Upper Sorbian #language-Haitian #language-Hungarian #language-Armenian #language-Western Armenian #language-Interlingua (International Auxiliary Language Association) #language-Indonesian #language-Interlingue #language-Igbo #language-Sichuan Yi #language-Inupiaq #language-Iloko #language-Ingush #language-Ido #language-Icelandic #language-Italian #language-Inuktitut #language-Japanese #language-Jamaican Creole English #language-Lojban #language-Javanese #language-Georgian #language-Kara-Kalpak #language-Kabyle #language-Kabardian #language-Kabiyè #language-Tyap #language-Kongo #language-Kikuyu #language-Kuanyama #language-Kazakh #language-Kalaallisut #language-Khmer #language-Kannada #language-Korean #language-Komi-Permyak #language-Karachay-Balkar #language-Kashmiri #language-Kölsch #language-Kurdish #language-Komi #language-Cornish #language-Kirghiz #language-Latin #language-Ladino #language-Luxembourgish #language-Lak #language-Lezghian #language-Lingua Franca Nova #language-Ganda #language-Limburgan #language-Ligurian #language-Ladin #language-Lombard #language-Lingala #language-Lao #language-Northern Luri #language-Lithuanian #language-Latgalian #language-Latvian #language-Madurese #language-Maithili #language-Moksha #language-Malagasy #language-Marshallese #language-Eastern Mari #language-Maori #language-Minangkabau #language-Macedonian #language-Malayalam #language-Mongolian #language-Manipuri #language-Mon #language-Marathi #language-Western Mari #language-Malay (macrolanguage) #language-Maltese #language-Creek #language-Mirandese #language-Burmese #language-Erzya #language-Mazanderani #language-nah #language-Neapolitan #language-Low German #language-Nepali (macrolanguage) #language-Newari #language-Ndonga #language-Nias #language-Dutch #language-Norwegian Nynorsk #language-Norwegian #language-Novial #language-N'Ko #language-Narom #language-Pedi #language-Navajo #language-Nyanja #language-Occitan (post 1500) #language-Livvi #language-Oromo #language-Oriya (macrolanguage) #language-Ossetian #language-Panjabi #language-Pangasinan #language-Pampanga #language-Papiamento #language-Picard #language-Nigerian Pidgin #language-Pennsylvania German #language-Pfaelzisch #language-Pali #language-Pitcairn-Norfolk #language-Polish #language-Piemontese #language-Western Panjabi #language-Pontic #language-Pushto #language-Portuguese #language-Paiwan #language-Quechua #language-Romansh #language-Vlax Romani #language-Rundi #language-Romanian #language-Russian #language-Rusyn #language-Kinyarwanda #language-Sanskrit #language-Yakut #language-Santali #language-Sardinian #language-Sicilian #language-Scots #language-Sindhi #language-Northern Sami #language-Sango #language-Serbo-Croatian #language-Tachelhit #language-Shan #language-Sinhala #language-Slovak #language-Saraiki #language-Slovenian #language-Samoan #language-Inari Sami #language-Shona #language-Somali #language-Albanian #language-Serbian #language-Sranan Tongo #language-Swati #language-Southern Sotho #language-Saterfriesisch #language-Sundanese #language-Swedish #language-Swahili (macrolanguage) #language-Silesian #language-Sakizaya #language-Tamil #language-Atayal #language-Tulu #language-Telugu #language-Tetum #language-Tajik #language-Thai #language-Tigrinya #language-Turkmen #language-Tagalog #language-Tswana #language-Tonga (Tonga Islands) #language-Tok Pisin #language-Turkish #language-Sediq #language-Tsonga #language-Tatar #language-Tumbuka #language-Twi #language-Tahitian #language-Tuvinian #language-Udmurt #language-Uighur #language-Ukrainian #language-Urdu #language-Uzbek #language-Venda #language-Venetian #language-Veps #language-Vietnamese #language-Vlaams #language-Volapük #language-Walloon #language-Waray (Philippines) #language-Wolof #language-Wu Chinese #language-Kalmyk #language-Xhosa #language-Mingrelian #language-Yiddish #language-Yoruba #language-Zhuang #language-Zeeuws #language-Chinese #language-Zulu #license-cc-by-sa-3.0 #license-gfdl #region-us 
 ### 1. Determine the date of the dump you are interested in


Choose one wikipedia dump, for instance <URL
and identify the date. ### 2. [Optional] Get a refreshed list of languages


This is optional because it not very likely that a new language will have
suddenly appeared since the last version *and* have a significant dataset.


Navigate to <URL and copy the
languages column from the "Detailed list" table (near the end of the page).


Copy that content in the form of a Python list into 'lang\_def.py' (at the top
of the repo) under a new date. ### 3. [Optional] Create Media and Category aliases


In order to properly extract links to images and media in all languages, we
must refresh the two corresponding files. To do so, from the root of the repo,
run


This will create or update these two files at the root of the repo:


* 'media\_aliases.py'
* 'category\_aliases.py'


These files are used in the final step ### 4. Build and prepare the datasets into sharded parquet files


Running this script downloads the wikipedia dumps for each language in
'lang\_def.py' and shards each language dataset into the appropriate number of
shards (max size ~ 250MB).


There are other options:


For instance, for faster downloads of the dumps, use the mirror option:


It will download the dumps at around 60MB/s instead of the capped speed
(~4MB/s) from . The script will skip existing
directories, allowing you to run the script in several passes.


Notes:


* These instructions build upon the build process of the
Wikipedia Dataset. HF did a
fantastic job, I just pushed it a bit further.
* Be aware that not all mirrors contain all dumps. For instance URL
does not contain dumps for languages such as be-x-old or cbk-zam. My own
solution is to run a first pass using the aforementioned mirror, and a second
pass with the official 'URL' site (omitting the
'--mirror' parameter).