This dataset repository contains the [TFRecord](https://www.tensorflow.org/tutorials/load_data/tfrecord) shards of the [WikText (v1) dataset](https://huggingface.co/datasets/wikitext.

We used [this script](https://github.com/huggingface/transformers/blob/main/examples/tensorflow/language-modeling-tpu/prepare_tfrecord_shards.py) to prepare these TFRecord shards.

For more details on how these TFRecord shards should be used, refer to the following tutorial: [Training a masked language model end-to-end from scratch on TPUs
](https://github.com/huggingface/transformers/tree/main/examples/tensorflow/language-modeling-tpu).

region:us
{}
TAGS
#region-us 

This dataset repository contains the TFRecord shards of the WikText (v1) dataset to prepare these TFRecord shards.

For more details on how these TFRecord shards should be used, refer to the following tutorial: Training a masked language model end-to-end from scratch on TPUs
.

TAGS
#region-us 
