
# 

The [`tatsu-lab/alpaca` dataset](https://huggingface.co/datasets/tatsu-lab/alpaca) was split into train/test/val with the goal of training text-to-text generation models to generate instruction prompts corresponding to arbitrary text.
To do this, you would use

- `output` as **the text2text model** input column
- `instruction` as the text2text model target/output column

## modifications & filtering 

Rows that used the column `input` in the original dataset, and rows where the `output` column contains less than 8 words were dropped.

Link to [function used to filter](https://gist.github.com/pszemraj/3633acb0cf3288d49b7bee550e756839) the original dataset after splitting
- The filter_dataset function reads datasets, counts tokens in specified columns, filters rows based on a minimum number of tokens, drops specified columns and/or rows with non-NaN values, and saves the modified datasets to a new directory. It returns summary statistics of the modified records.

## dataset info


Output of loading the dataset:

```python
DatasetDict({
    train: Dataset({
        features: ['instruction', 'output'],
        num_rows: 23167
    })
    test: Dataset({
        features: ['instruction', 'output'],
        num_rows: 2822
    })
    validation: Dataset({
        features: ['instruction', 'output'],
        num_rows: 2866
    })
})
```


## token counts in the `output` column


t5 

![output-tokens](https://i.imgur.com/cOCeBmA.png)


bart-base 

![output-tokens](https://i.imgur.com/y5CZiMa.png)


---
task_categories:text-generation task_categories:text2text-generation size_categories:10K<n<100K source_datasets:tatsu-lab/alpaca language:en license:cc-by-4.0 alpaca instruction generation region:us
{"language": ["en"], "license": "cc-by-4.0", "size_categories": ["10K<n<100K"], "source_datasets": "tatsu-lab/alpaca", "task_categories": ["text-generation", "text2text-generation"], "tags": ["alpaca", "instruction generation"]}
TAGS
#task_categories-text-generation #task_categories-text2text-generation #size_categories-10K<n<100K #source_datasets-tatsu-lab/alpaca #language-English #license-cc-by-4.0 #alpaca #instruction generation #region-us 


# 

The 'tatsu-lab/alpaca' dataset was split into train/test/val with the goal of training text-to-text generation models to generate instruction prompts corresponding to arbitrary text.
To do this, you would use

- 'output' as the text2text model input column
- 'instruction' as the text2text model target/output column

## modifications & filtering 

Rows that used the column 'input' in the original dataset, and rows where the 'output' column contains less than 8 words were dropped.

Link to function used to filter the original dataset after splitting
- The filter_dataset function reads datasets, counts tokens in specified columns, filters rows based on a minimum number of tokens, drops specified columns and/or rows with non-NaN values, and saves the modified datasets to a new directory. It returns summary statistics of the modified records.

## dataset info


Output of loading the dataset:




## token counts in the 'output' column


t5 

!output-tokens


bart-base 

!output-tokens


---
# 

The 'tatsu-lab/alpaca' dataset was split into train/test/val with the goal of training text-to-text generation models to generate instruction prompts corresponding to arbitrary text.
To do this, you would use

- 'output' as the text2text model input column
- 'instruction' as the text2text model target/output column ## modifications & filtering 

Rows that used the column 'input' in the original dataset, and rows where the 'output' column contains less than 8 words were dropped.

Link to function used to filter the original dataset after splitting
- The filter_dataset function reads datasets, counts tokens in specified columns, filters rows based on a minimum number of tokens, drops specified columns and/or rows with non-NaN values, and saves the modified datasets to a new directory. It returns summary statistics of the modified records. ## dataset info


Output of loading the dataset: ## token counts in the 'output' column


t5 

!output-tokens


bart-base 

!output-tokens


---
TAGS
#task_categories-text-generation #task_categories-text2text-generation #size_categories-10K<n<100K #source_datasets-tatsu-lab/alpaca #language-English #license-cc-by-4.0 #alpaca #instruction generation #region-us 
 # 

The 'tatsu-lab/alpaca' dataset was split into train/test/val with the goal of training text-to-text generation models to generate instruction prompts corresponding to arbitrary text.
To do this, you would use

- 'output' as the text2text model input column
- 'instruction' as the text2text model target/output column ## modifications & filtering 

Rows that used the column 'input' in the original dataset, and rows where the 'output' column contains less than 8 words were dropped.

Link to function used to filter the original dataset after splitting
- The filter_dataset function reads datasets, counts tokens in specified columns, filters rows based on a minimum number of tokens, drops specified columns and/or rows with non-NaN values, and saves the modified datasets to a new directory. It returns summary statistics of the modified records. ## dataset info


Output of loading the dataset: ## token counts in the 'output' column


t5 

!output-tokens


bart-base 

!output-tokens


---