

# text2image multi-prompt(s): a dataset collection

- collection of several text2image prompt datasets
- data was cleaned/normalized with the goal of removing "model specific APIs" like the "--ar" for Midjourney and so on
- data de-duplicated on a basic level: exactly duplicate prompts were dropped (_after cleaning and normalization_)

## updates

- Oct 2023: the `default` config has been updated with better deduplication. It was deduplicated with minhash (_params: n-gram size set to 3, deduplication threshold at 0.6, hash function chosen as xxh3 with 32-bit hash bits, and 128 permutations with a batch size of 10,000._) which drops 2+ million rows.
  - original version is still available under `config_name="original"`

## contents



default:

```
DatasetDict({
    train: Dataset({
        features: ['text', 'src_dataset'],
        num_rows: 1677221
    })
    test: Dataset({
        features: ['text', 'src_dataset'],
        num_rows: 292876
    })
})
```

For `original` config:
```
DatasetDict({
    train: Dataset({
        features: ['text', 'src_dataset'],
        num_rows: 3551734
    })
    test: Dataset({
        features: ['text', 'src_dataset'],
        num_rows: 399393
    })
})
```

_NOTE: as the other two datasets did not have a `validation` split, the validation split of `succinctly/midjourney-prompts` was merged into `train`._
task_categories:text-generation task_categories:feature-extraction multilinguality:monolingual source_datasets:bartman081523/stable-diffusion-discord-prompts source_datasets:succinctly/midjourney-prompts source_datasets:Gustavosta/Stable-Diffusion-Prompts language:en license:apache-2.0 text generation region:us
{"language": ["en"], "license": "apache-2.0", "multilinguality": ["monolingual"], "source_datasets": ["bartman081523/stable-diffusion-discord-prompts", "succinctly/midjourney-prompts", "Gustavosta/Stable-Diffusion-Prompts"], "task_categories": ["text-generation", "feature-extraction"], "pretty_name": "multi text2image prompts a dataset collection", "tags": ["text generation"], "configs": [{"config_name": "default", "data_files": [{"split": "train", "path": "data/train-*"}, {"split": "test", "path": "data/test-*"}]}, {"config_name": "original", "data_files": [{"split": "train", "path": "original/train-*"}, {"split": "test", "path": "original/test-*"}]}], "dataset_info": [{"config_name": "default", "features": [{"name": "text", "dtype": "string"}, {"name": "src_dataset", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 262736830, "num_examples": 1677221}, {"name": "test", "num_bytes": 56294291, "num_examples": 292876}], "download_size": 151054782, "dataset_size": 319031121}, {"config_name": "original", "features": [{"name": "text", "dtype": "string"}, {"name": "src_dataset", "dtype": "string"}], "splits": [{"name": "train", "num_bytes": 741427383, "num_examples": 3551734}, {"name": "test", "num_bytes": 83615440, "num_examples": 399393}], "download_size": 402186258, "dataset_size": 825042823}]}
TAGS
#task_categories-text-generation #task_categories-feature-extraction #multilinguality-monolingual #source_datasets-bartman081523/stable-diffusion-discord-prompts #source_datasets-succinctly/midjourney-prompts #source_datasets-Gustavosta/Stable-Diffusion-Prompts #language-English #license-apache-2.0 #text generation #region-us 



# text2image multi-prompt(s): a dataset collection

- collection of several text2image prompt datasets
- data was cleaned/normalized with the goal of removing "model specific APIs" like the "--ar" for Midjourney and so on
- data de-duplicated on a basic level: exactly duplicate prompts were dropped (_after cleaning and normalization_)

## updates

- Oct 2023: the 'default' config has been updated with better deduplication. It was deduplicated with minhash (_params: n-gram size set to 3, deduplication threshold at 0.6, hash function chosen as xxh3 with 32-bit hash bits, and 128 permutations with a batch size of 10,000._) which drops 2+ million rows.
  - original version is still available under 'config_name="original"'

## contents



default:



For 'original' config:


_NOTE: as the other two datasets did not have a 'validation' split, the validation split of 'succinctly/midjourney-prompts' was merged into 'train'._
# text2image multi-prompt(s): a dataset collection

- collection of several text2image prompt datasets
- data was cleaned/normalized with the goal of removing "model specific APIs" like the "--ar" for Midjourney and so on
- data de-duplicated on a basic level: exactly duplicate prompts were dropped (_after cleaning and normalization_) ## updates

- Oct 2023: the 'default' config has been updated with better deduplication. It was deduplicated with minhash (_params: n-gram size set to 3, deduplication threshold at 0.6, hash function chosen as xxh3 with 32-bit hash bits, and 128 permutations with a batch size of 10,000._) which drops 2+ million rows.
  - original version is still available under 'config_name="original"' ## contents



default:



For 'original' config:


_NOTE: as the other two datasets did not have a 'validation' split, the validation split of 'succinctly/midjourney-prompts' was merged into 'train'._
TAGS
#task_categories-text-generation #task_categories-feature-extraction #multilinguality-monolingual #source_datasets-bartman081523/stable-diffusion-discord-prompts #source_datasets-succinctly/midjourney-prompts #source_datasets-Gustavosta/Stable-Diffusion-Prompts #language-English #license-apache-2.0 #text generation #region-us 
 # text2image multi-prompt(s): a dataset collection

- collection of several text2image prompt datasets
- data was cleaned/normalized with the goal of removing "model specific APIs" like the "--ar" for Midjourney and so on
- data de-duplicated on a basic level: exactly duplicate prompts were dropped (_after cleaning and normalization_) ## updates

- Oct 2023: the 'default' config has been updated with better deduplication. It was deduplicated with minhash (_params: n-gram size set to 3, deduplication threshold at 0.6, hash function chosen as xxh3 with 32-bit hash bits, and 128 permutations with a batch size of 10,000._) which drops 2+ million rows.
  - original version is still available under 'config_name="original"' ## contents



default:



For 'original' config:


_NOTE: as the other two datasets did not have a 'validation' split, the validation split of 'succinctly/midjourney-prompts' was merged into 'train'._