
# Dataset Card for unconstrained_language

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
- [Citation Information](#additional-information)
  - [Citation Information](#citation-information)


## Dataset Description

- **Paper: https://arxiv.org/abs/2305.17266**
- **Point of Contact: vijeta_deshpande@student.uml.edu**

### Dataset Summary

This dataset is one of the two datasets published by "Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale" (https://arxiv.org/abs/2305.17266). 
The dataset available at this link is the pre-training data **not** constrained by any predefined vocabulary. The other published data i.e. the pre-training data that is constrained by vocabulary is available at https://huggingface.co/datasets/text-machine-lab/constrained_language.

This dataset is curated by randomly sampling text spans (of an approximate length of 128 tokens) from the following corpora,
- C4: https://arxiv.org/abs/1910.10683,
- BookCorpus: https://ieeexplore.ieee.org/document/7410368,
- Wikipedia: https://huggingface.co/datasets/wikipedia,
- Simplified-Wikipedia: https://simple.wikipedia.org/wiki/Main_Page,
- Children's Book Test Corpus: https://arxiv.org/abs/1511.02301

The dataset includes ~9 million contiguous spans, each with approximately 128 tokens. 

### Languages

The dataset contains the English language only.

## Dataset Structure

The dataset is available in the arrow dataset format with three splits namely, train, validation, and test. Every data instance has only one key "Text" that included a text span of approximately 128 tokens.

### Citation Information
If this dataset is useful to you please cite our work.

```sh
@article{deshpande2023honey,
  title={Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale},
  author={Deshpande, Vijeta and Pechi, Dan and Thatte, Shree and Lialin, Vladislav and Rumshisky, Anna},
  journal={arXiv preprint arXiv:2305.17266},
  year={2023}
}
```
arxiv:2305.17266 arxiv:1910.10683 arxiv:1511.02301 region:us
{"dataset_info": {"features": [{"name": "TEXT", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 5437652389, "num_examples": 9081490}, {"name": "validation", "num_bytes": 50107745, "num_examples": 100000}, {"name": "test", "num_bytes": 50134861, "num_examples": 100000}], "download_size": 3732550490, "dataset_size": 5537894995}}
TAGS
#arxiv-2305.17266 #arxiv-1910.10683 #arxiv-1511.02301 #region-us 


# Dataset Card for unconstrained_language

## Table of Contents
- Dataset Description
  - Dataset Summary
  - Languages
- Dataset Structure
  - Data Instances
- Citation Information
  - Citation Information


## Dataset Description

- Paper: URL
- Point of Contact: vijeta_deshpande@URL

### Dataset Summary

This dataset is one of the two datasets published by "Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale" (URL 
The dataset available at this link is the pre-training data not constrained by any predefined vocabulary. The other published data i.e. the pre-training data that is constrained by vocabulary is available at URL

This dataset is curated by randomly sampling text spans (of an approximate length of 128 tokens) from the following corpora,
- C4: URL
- BookCorpus: URL
- Wikipedia: URL
- Simplified-Wikipedia: URL
- Children's Book Test Corpus: URL

The dataset includes ~9 million contiguous spans, each with approximately 128 tokens. 

### Languages

The dataset contains the English language only.

## Dataset Structure

The dataset is available in the arrow dataset format with three splits namely, train, validation, and test. Every data instance has only one key "Text" that included a text span of approximately 128 tokens.


If this dataset is useful to you please cite our work.


# Dataset Card for unconstrained_language ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Languages
- Dataset Structure
  - Data Instances
- Citation Information
  - Citation Information ## Dataset Description

- Paper: URL
- Point of Contact: vijeta_deshpande@URL ### Dataset Summary

This dataset is one of the two datasets published by "Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale" (URL 
The dataset available at this link is the pre-training data not constrained by any predefined vocabulary. The other published data i.e. the pre-training data that is constrained by vocabulary is available at URL

This dataset is curated by randomly sampling text spans (of an approximate length of 128 tokens) from the following corpora,
- C4: URL
- BookCorpus: URL
- Wikipedia: URL
- Simplified-Wikipedia: URL
- Children's Book Test Corpus: URL

The dataset includes ~9 million contiguous spans, each with approximately 128 tokens. ### Languages

The dataset contains the English language only. ## Dataset Structure

The dataset is available in the arrow dataset format with three splits namely, train, validation, and test. Every data instance has only one key "Text" that included a text span of approximately 128 tokens.


If this dataset is useful to you please cite our work.
TAGS
#arxiv-2305.17266 #arxiv-1910.10683 #arxiv-1511.02301 #region-us 
 # Dataset Card for unconstrained_language ## Table of Contents
- Dataset Description
  - Dataset Summary
  - Languages
- Dataset Structure
  - Data Instances
- Citation Information
  - Citation Information ## Dataset Description

- Paper: URL
- Point of Contact: vijeta_deshpande@URL ### Dataset Summary

This dataset is one of the two datasets published by "Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale" (URL 
The dataset available at this link is the pre-training data not constrained by any predefined vocabulary. The other published data i.e. the pre-training data that is constrained by vocabulary is available at URL

This dataset is curated by randomly sampling text spans (of an approximate length of 128 tokens) from the following corpora,
- C4: URL
- BookCorpus: URL
- Wikipedia: URL
- Simplified-Wikipedia: URL
- Children's Book Test Corpus: URL

The dataset includes ~9 million contiguous spans, each with approximately 128 tokens. ### Languages

The dataset contains the English language only. ## Dataset Structure

The dataset is available in the arrow dataset format with three splits namely, train, validation, and test. Every data instance has only one key "Text" that included a text span of approximately 128 tokens.


If this dataset is useful to you please cite our work.