
# FolkGPT

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on vicclab/fairy_tales dataset.

## Model description

This model is the result of fine-tuning gpt2 on a dataset of fairy tales from various cultures.

## Intended uses & limitations

The idea behind this is to generate text in the fashion of fairy tales written in the 18th and 19th centuries.

Why? Fairy tales seemed an appropriate application for text generation, as stories are usually short(ish), 
self-contained, and easy to read. 

## Training and evaluation data

Trained on the vicclab/fairy_tales dataset. The dataset consists of a number of texts which 
were downloaded from Project Gutenberg, and then edited to remove all text except for the 
stories themselves. These were then all concatenated into a text file and pushed to HF at 
https://huggingface.co/datasets/vicclab/fairy_tales. The latest update to the dataset, which 
was used in the training of this model, was created and uploaded on February 26th, 2023.
Texts used [and token count after removing boilerplate text]:
https://www.gutenberg.org/files/2591/2591-0.txt [102927 tokens]
https://www.gutenberg.org/files/503/503-0.txt [138353 tokens]
https://www.gutenberg.org/cache/epub/69739/pg69739.txt [51035 tokens]
https://www.gutenberg.org/files/2435/2435-0.txt [98791 tokens]
https://www.gutenberg.org/cache/epub/7871/pg7871.txt [49410 tokens] 
https://www.gutenberg.org/files/8933/8933-0.txt [178622 tokens]
gutenberg.org/cache/epub/30834/pg30834.txt [58359 tokens]
https://www.gutenberg.org/cache/epub/68589/pg68589.txt [39815 tokens]
https://www.gutenberg.org/cache/epub/34453/pg34453.txt [69365 tokens]
gutenberg.org/cache/epub/8653/pg8653.txt [35351]

[Total tokens in actual dataset: 1002654 tokens]


## Training procedure
The dataset was loaded, sampling by paragraph. From here, the dataset was split into a training dataset 
and a validation dataset in an 80-20 split. These were then tokenized. The model was set up, and the trainer 
was instantiated with the training_arguments listed below. Then, the training took place.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 1
- mixed_precision_training: Native AMP

### Training results



### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.0
- Tokenizers 0.13.2
{"language": ["en"], "license": "mit", "tags": ["generated_from_trainer"], "datasets": ["vicclab/fairy_tales"], "pipeline_tag": "text-generation", "model-index": [{"name": "FolkGPT", "results": []}]}
transformers pytorch tensorboard gpt2 text-generation generated_from_trainer en dataset:vicclab/fairy_tales license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #tensorboard #gpt2 #text-generation #generated_from_trainer #en #dataset-vicclab/fairy_tales #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


# FolkGPT

This model is a fine-tuned version of gpt2 on vicclab/fairy_tales dataset.

## Model description

This model is the result of fine-tuning gpt2 on a dataset of fairy tales from various cultures.

## Intended uses & limitations

The idea behind this is to generate text in the fashion of fairy tales written in the 18th and 19th centuries.

Why? Fairy tales seemed an appropriate application for text generation, as stories are usually short(ish), 
self-contained, and easy to read. 

## Training and evaluation data

Trained on the vicclab/fairy_tales dataset. The dataset consists of a number of texts which 
were downloaded from Project Gutenberg, and then edited to remove all text except for the 
stories themselves. These were then all concatenated into a text file and pushed to HF at 
URL The latest update to the dataset, which 
was used in the training of this model, was created and uploaded on February 26th, 2023.
Texts used [and token count after removing boilerplate text]:
URL [102927 tokens]
URL [138353 tokens]
URL [51035 tokens]
URL [98791 tokens]
URL [49410 tokens] 
URL [178622 tokens]
URL [58359 tokens]
URL [39815 tokens]
URL [69365 tokens]
URL [35351]

[Total tokens in actual dataset: 1002654 tokens]


## Training procedure
The dataset was loaded, sampling by paragraph. From here, the dataset was split into a training dataset 
and a validation dataset in an 80-20 split. These were then tokenized. The model was set up, and the trainer 
was instantiated with the training_arguments listed below. Then, the training took place.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 1
- mixed_precision_training: Native AMP

### Training results



### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.0
- Tokenizers 0.13.2
# FolkGPT

This model is a fine-tuned version of gpt2 on vicclab/fairy_tales dataset. ## Model description

This model is the result of fine-tuning gpt2 on a dataset of fairy tales from various cultures. ## Intended uses & limitations

The idea behind this is to generate text in the fashion of fairy tales written in the 18th and 19th centuries.

Why? Fairy tales seemed an appropriate application for text generation, as stories are usually short(ish), 
self-contained, and easy to read. ## Training and evaluation data

Trained on the vicclab/fairy_tales dataset. The dataset consists of a number of texts which 
were downloaded from Project Gutenberg, and then edited to remove all text except for the 
stories themselves. These were then all concatenated into a text file and pushed to HF at 
URL The latest update to the dataset, which 
was used in the training of this model, was created and uploaded on February 26th, 2023.
Texts used [and token count after removing boilerplate text]:
URL [102927 tokens]
URL [138353 tokens]
URL [51035 tokens]
URL [98791 tokens]
URL [49410 tokens] 
URL [178622 tokens]
URL [58359 tokens]
URL [39815 tokens]
URL [69365 tokens]
URL [35351]

[Total tokens in actual dataset: 1002654 tokens] ## Training procedure
The dataset was loaded, sampling by paragraph. From here, the dataset was split into a training dataset 
and a validation dataset in an 80-20 split. These were then tokenized. The model was set up, and the trainer 
was instantiated with the training_arguments listed below. Then, the training took place. ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 1
- mixed_precision_training: Native AMP ### Training results ### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.0
- Tokenizers 0.13.2
TAGS
#transformers #pytorch #tensorboard #gpt2 #text-generation #generated_from_trainer #en #dataset-vicclab/fairy_tales #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # FolkGPT

This model is a fine-tuned version of gpt2 on vicclab/fairy_tales dataset. ## Model description

This model is the result of fine-tuning gpt2 on a dataset of fairy tales from various cultures. ## Intended uses & limitations

The idea behind this is to generate text in the fashion of fairy tales written in the 18th and 19th centuries.

Why? Fairy tales seemed an appropriate application for text generation, as stories are usually short(ish), 
self-contained, and easy to read. ## Training and evaluation data

Trained on the vicclab/fairy_tales dataset. The dataset consists of a number of texts which 
were downloaded from Project Gutenberg, and then edited to remove all text except for the 
stories themselves. These were then all concatenated into a text file and pushed to HF at 
URL The latest update to the dataset, which 
was used in the training of this model, was created and uploaded on February 26th, 2023.
Texts used [and token count after removing boilerplate text]:
URL [102927 tokens]
URL [138353 tokens]
URL [51035 tokens]
URL [98791 tokens]
URL [49410 tokens] 
URL [178622 tokens]
URL [58359 tokens]
URL [39815 tokens]
URL [69365 tokens]
URL [35351]

[Total tokens in actual dataset: 1002654 tokens] ## Training procedure
The dataset was loaded, sampling by paragraph. From here, the dataset was split into a training dataset 
and a validation dataset in an 80-20 split. These were then tokenized. The model was set up, and the trainer 
was instantiated with the training_arguments listed below. Then, the training took place. ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 1
- mixed_precision_training: Native AMP ### Training results ### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.0
- Tokenizers 0.13.2
TAGS
#transformers #pytorch #tensorboard #gpt2 #text-generation #generated_from_trainer #en #dataset-vicclab/fairy_tales #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# FolkGPT

This model is a fine-tuned version of gpt2 on vicclab/fairy_tales dataset.## Model description

This model is the result of fine-tuning gpt2 on a dataset of fairy tales from various cultures.## Intended uses & limitations

The idea behind this is to generate text in the fashion of fairy tales written in the 18th and 19th centuries.

Why? Fairy tales seemed an appropriate application for text generation, as stories are usually short(ish), 
self-contained, and easy to read.## Training and evaluation data

Trained on the vicclab/fairy_tales dataset. The dataset consists of a number of texts which 
were downloaded from Project Gutenberg, and then edited to remove all text except for the 
stories themselves. These were then all concatenated into a text file and pushed to HF at 
URL The latest update to the dataset, which 
was used in the training of this model, was created and uploaded on February 26th, 2023.
Texts used [and token count after removing boilerplate text]:
URL [102927 tokens]
URL [138353 tokens]
URL [51035 tokens]
URL [98791 tokens]
URL [49410 tokens] 
URL [178622 tokens]
URL [58359 tokens]
URL [39815 tokens]
URL [69365 tokens]
URL [35351]

[Total tokens in actual dataset: 1002654 tokens]## Training procedure
The dataset was loaded, sampling by paragraph. From here, the dataset was split into a training dataset 
and a validation dataset in an 80-20 split. These were then tokenized. The model was set up, and the trainer 
was instantiated with the training_arguments listed below. Then, the training took place.### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 1
- mixed_precision_training: Native AMP### Training results### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.0
- Tokenizers 0.13.2