
# Pyg-VN-2.7B

This model is a fine-tuned version of [PygmalionAI/pygmalion-2.7b](https://huggingface.co/PygmalionAI/pygmalion-2.7b).
The base model is [GPT-Neo-2.7B](https://huggingface.co/EleutherAI/gpt-neo-2.7B), which was fine-tuned with a dataset of Discord/Reddit conversations by Haru in [this commit](https://huggingface.co/hakurei/convogpt/tree/07707377dee0aa7d1ee5363ef660b13eb5b73f9d/2.7b-uft) and was later fine-tuned by the PygmalionAI team on various conversational data.

## Training Data

The data used for training this revision is about 59M tokens (189MB) of Visual Novel data. 

## Intended uses and limitations

This model was trained for research and evaluation purposes to observe what effects Visual Novels would have on a chat-bot model.
It has the potential to output X-rated content.

## Training Procedure and Hardware

The model was fine-tuned on a single A100-80GB with the following settings:

```json
{
    "Batch size": 1,
    "Epoch": 1,
    "FP16 Level": O2,
    "Gradient Steps": 8,
    "Learning Rate": 5e-5,
    "Adam_Beta1": 0.9,
    "Adam_Beta2": 0.98,
    "Weight Decay": 0.1,
    "Block Size": 2048
}

```

Training Results:
```json
{
    "epoch": 1.0,
    "train_loss": 2.0835590305339813,
    "train_runtime": 14124.2003,
    "train_samples": 26672,
    "train_samples_per_second": 1.888,
    "train_steps_per_second": 0.236
}
```

Evaluation Results:

```json
{
    "epoch": 1.0,
    "eval_accuracy": 0.556417810103348,
    "eval_loss": 2.0116419792175293,
    "eval_runtime": 317.7849,
    "eval_samples": 1648,
    "eval_samples_per_second": 5.186,
    "eval_steps_per_second": 5.186,
    "perplexity": 7.4755820257152745
}
```

## How to use
You can use either [KoboldAI](https://github.com/henk717/KoboldAI) or [Text-Generation-WebUI](https://github.com/oobabooga/text-generation-webui) to load the model. 

You can also load the model directly with a pipeline for textgen:

```python
from transformers import pipeline
generator = pipeline('text-generation', model='AlpinDale/pyg-vn-2.7b')
generator("Last summer we", do_sample=True, min_length=50)
```

## Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu116
- Datasets 2.4.0
- Tokenizers 0.12.1
{"language": ["en"], "license": "creativeml-openrail-m", "pipeline_tag": "text-generation"}
transformers pytorch gpt_neo text-generation en license:creativeml-openrail-m autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #gpt_neo #text-generation #en #license-creativeml-openrail-m #autotrain_compatible #endpoints_compatible #region-us 


# Pyg-VN-2.7B

This model is a fine-tuned version of PygmalionAI/pygmalion-2.7b.
The base model is GPT-Neo-2.7B, which was fine-tuned with a dataset of Discord/Reddit conversations by Haru in this commit and was later fine-tuned by the PygmalionAI team on various conversational data.

## Training Data

The data used for training this revision is about 59M tokens (189MB) of Visual Novel data. 

## Intended uses and limitations

This model was trained for research and evaluation purposes to observe what effects Visual Novels would have on a chat-bot model.
It has the potential to output X-rated content.

## Training Procedure and Hardware

The model was fine-tuned on a single A100-80GB with the following settings:



Training Results:


Evaluation Results:



## How to use
You can use either KoboldAI or Text-Generation-WebUI to load the model. 

You can also load the model directly with a pipeline for textgen:



## Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu116
- Datasets 2.4.0
- Tokenizers 0.12.1
# Pyg-VN-2.7B

This model is a fine-tuned version of PygmalionAI/pygmalion-2.7b.
The base model is GPT-Neo-2.7B, which was fine-tuned with a dataset of Discord/Reddit conversations by Haru in this commit and was later fine-tuned by the PygmalionAI team on various conversational data. ## Training Data

The data used for training this revision is about 59M tokens (189MB) of Visual Novel data. ## Intended uses and limitations

This model was trained for research and evaluation purposes to observe what effects Visual Novels would have on a chat-bot model.
It has the potential to output X-rated content. ## Training Procedure and Hardware

The model was fine-tuned on a single A100-80GB with the following settings:



Training Results:


Evaluation Results: ## How to use
You can use either KoboldAI or Text-Generation-WebUI to load the model. 

You can also load the model directly with a pipeline for textgen: ## Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu116
- Datasets 2.4.0
- Tokenizers 0.12.1
TAGS
#transformers #pytorch #gpt_neo #text-generation #en #license-creativeml-openrail-m #autotrain_compatible #endpoints_compatible #region-us 
 # Pyg-VN-2.7B

This model is a fine-tuned version of PygmalionAI/pygmalion-2.7b.
The base model is GPT-Neo-2.7B, which was fine-tuned with a dataset of Discord/Reddit conversations by Haru in this commit and was later fine-tuned by the PygmalionAI team on various conversational data. ## Training Data

The data used for training this revision is about 59M tokens (189MB) of Visual Novel data. ## Intended uses and limitations

This model was trained for research and evaluation purposes to observe what effects Visual Novels would have on a chat-bot model.
It has the potential to output X-rated content. ## Training Procedure and Hardware

The model was fine-tuned on a single A100-80GB with the following settings:



Training Results:


Evaluation Results: ## How to use
You can use either KoboldAI or Text-Generation-WebUI to load the model. 

You can also load the model directly with a pipeline for textgen: ## Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu116
- Datasets 2.4.0
- Tokenizers 0.12.1
TAGS
#transformers #pytorch #gpt_neo #text-generation #en #license-creativeml-openrail-m #autotrain_compatible #endpoints_compatible #region-us 
# Pyg-VN-2.7B

This model is a fine-tuned version of PygmalionAI/pygmalion-2.7b.
The base model is GPT-Neo-2.7B, which was fine-tuned with a dataset of Discord/Reddit conversations by Haru in this commit and was later fine-tuned by the PygmalionAI team on various conversational data.## Training Data

The data used for training this revision is about 59M tokens (189MB) of Visual Novel data.## Intended uses and limitations

This model was trained for research and evaluation purposes to observe what effects Visual Novels would have on a chat-bot model.
It has the potential to output X-rated content.## Training Procedure and Hardware

The model was fine-tuned on a single A100-80GB with the following settings:



Training Results:


Evaluation Results:## How to use
You can use either KoboldAI or Text-Generation-WebUI to load the model. 

You can also load the model directly with a pipeline for textgen:## Framework versions

- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu116
- Datasets 2.4.0
- Tokenizers 0.12.1