
Original Dataset [JeanKaddour/minipile](https://huggingface.co/datasets/JeanKaddour/minipile)


See the [Thought Tokens Repository](https://github.com/ZelaAI/thought-tokens) for demonstration of streaming usage of this dataset and specific implementation of how this dataset was prepared.

Tokenized with the GPTNeoX tokenizer, split into sequences of length 513, intended for 512 input and 512 target ids.
task_categories:text-generation size_categories:1B<n<10B region:us
{"size_categories": ["1B<n<10B"], "task_categories": ["text-generation"], "pretty_name": "Streamable Minipile Tokenized"}
TAGS
#task_categories-text-generation #size_categories-1B<n<10B #region-us 


Original Dataset JeanKaddour/minipile


See the Thought Tokens Repository for demonstration of streaming usage of this dataset and specific implementation of how this dataset was prepared.

Tokenized with the GPTNeoX tokenizer, split into sequences of length 513, intended for 512 input and 512 target ids.
TAGS
#task_categories-text-generation #size_categories-1B<n<10B #region-us 

passage: TAGS
#task_categories-text-generation #size_categories-1B<n<10B #region-us 
