# CC-100 zh-Hant (Traditional Chinese)

From https://data.statmt.org/cc-100/, only zh-Hant - Chinese (Traditional). Broken into lines, with each line as a row.

Estimated to have around 4B tokens when tokenized with the [`bigscience/bloom`](https://huggingface.co/bigscience/bloom) tokenizer.

There's another version that the text is split by paragraphs instead of lines: [`zetavg/CC-100-zh-Hant-merged`](https://huggingface.co/datasets/zetavg/CC-100-zh-Hant-merged).

## References

Please cite the following if you found the resources in the CC-100 corpus useful.

* **Unsupervised Cross-lingual Representation Learning at Scale**, *Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov*, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), p. 8440-8451, July 2020, [pdf](https://www.aclweb.org/anthology/2020.acl-main.747.pdf), [bib](https://www.aclweb.org/anthology/2020.acl-main.747.bib) .
* **CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data**, *Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, Edouard Grave*, Proceedings of the 12th Language Resources and Evaluation Conference (LREC), p. 4003-4012, May 2020,  [pdf](https://www.aclweb.org/anthology/2020.lrec-1.494.pdf), [bib](https://www.aclweb.org/anthology/2020.lrec-1.494.bib).
task_categories:text-generation language:zh region:us
{"language": ["zh"], "task_categories": ["text-generation"], "dataset_info": {"features": [{"name": "line", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 18198166302, "num_examples": 85165683}], "download_size": 13296002208, "dataset_size": 18198166302}}
TAGS
#task_categories-text-generation #language-Chinese #region-us 

# CC-100 zh-Hant (Traditional Chinese)

From URL only zh-Hant - Chinese (Traditional). Broken into lines, with each line as a row.

Estimated to have around 4B tokens when tokenized with the 'bigscience/bloom' tokenizer.

There's another version that the text is split by paragraphs instead of lines: 'zetavg/CC-100-zh-Hant-merged'.

## References

Please cite the following if you found the resources in the CC-100 corpus useful.

* Unsupervised Cross-lingual Representation Learning at Scale, *Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov*, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), p. 8440-8451, July 2020, pdf, bib .
* CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data, *Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, Edouard Grave*, Proceedings of the 12th Language Resources and Evaluation Conference (LREC), p. 4003-4012, May 2020,  pdf, bib.
# CC-100 zh-Hant (Traditional Chinese)

From URL only zh-Hant - Chinese (Traditional). Broken into lines, with each line as a row.

Estimated to have around 4B tokens when tokenized with the 'bigscience/bloom' tokenizer.

There's another version that the text is split by paragraphs instead of lines: 'zetavg/CC-100-zh-Hant-merged'. ## References

Please cite the following if you found the resources in the CC-100 corpus useful.

* Unsupervised Cross-lingual Representation Learning at Scale, *Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov*, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), p. 8440-8451, July 2020, pdf, bib .
* CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data, *Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, Edouard Grave*, Proceedings of the 12th Language Resources and Evaluation Conference (LREC), p. 4003-4012, May 2020,  pdf, bib.
TAGS
#task_categories-text-generation #language-Chinese #region-us 
 # CC-100 zh-Hant (Traditional Chinese)

From URL only zh-Hant - Chinese (Traditional). Broken into lines, with each line as a row.

Estimated to have around 4B tokens when tokenized with the 'bigscience/bloom' tokenizer.

There's another version that the text is split by paragraphs instead of lines: 'zetavg/CC-100-zh-Hant-merged'. ## References

Please cite the following if you found the resources in the CC-100 corpus useful.

* Unsupervised Cross-lingual Representation Learning at Scale, *Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov*, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), p. 8440-8451, July 2020, pdf, bib .
* CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data, *Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, Edouard Grave*, Proceedings of the 12th Language Resources and Evaluation Conference (LREC), p. 4003-4012, May 2020,  pdf, bib.
passage: TAGS
#task_categories-text-generation #language-Chinese #region-us 
# CC-100 zh-Hant (Traditional Chinese)

From URL only zh-Hant - Chinese (Traditional). Broken into lines, with each line as a row.

Estimated to have around 4B tokens when tokenized with the 'bigscience/bloom' tokenizer.

There's another version that the text is split by paragraphs instead of lines: 'zetavg/CC-100-zh-Hant-merged'.## References

Please cite the following if you found the resources in the CC-100 corpus useful.

* Unsupervised Cross-lingual Representation Learning at Scale, *Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov*, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), p. 8440-8451, July 2020, pdf, bib .
* CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data, *Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, Edouard Grave*, Proceedings of the 12th Language Resources and Evaluation Conference (LREC), p. 4003-4012, May 2020,  pdf, bib.