transformers safetensors mistral text-generation dataset:ambrosfitz/mighty-history-merge dataset:ambrosfitz/textbook-openstax-yawp-merge license:cc autotrain_compatible endpoints_compatible text-generation-inference region:us
---
library_name: transformers
license: cc
datasets:
- ambrosfitz/mighty-history-merge
- ambrosfitz/textbook-openstax-yawp-merge
---

# Model Card for Model ID

An updated version of Neural History Chat, using the mighty-history-merge dataset to fine-tune the previous version (v1.0).

## Model Details
```
Run history:

train/epoch	▁▁▂▂▃▃▃▄▄▅▅▅▆▆▇▇▇██
train/global_step	▁▁▂▂▃▃▃▄▄▅▅▅▆▆▇▇▇██
train/learning_rate	▂▃▅▆▇█▇▇▆▆▅▄▄▃▃▂▂▁
train/loss	█▆▄▃▃▃▃▃▂▃▂▂▁▁▁▁▂▁
train/total_flos	▁
train/train_loss	▁
train/train_runtime	▁
train/train_samples_per_second	▁
train/train_steps_per_second	▁

Run summary:

train/epoch	1.98
train/global_step	92
train/learning_rate	0.0
train/loss	0.7792
train/total_flos	1.756453697101824e+16
train/train_loss	1.30356
train/train_runtime	1176.2194
train/train_samples_per_second	10.068
train/train_steps_per_second	0.078
```
## Training Explained

We went with a shorter training session of roughly 2 epochs for testing and evaluation. More steps/epochs might be
in the future, but colab pricing is pretty steep. Currently to merge the peft back to the model, requires roughly 40GB of
GPU RAM. So renting a Google Colab A100 is required and runs through credits quickly.