generated_from_trainer base_model:Aharneish/gpt2-spiritual license:mit region:us
---
license: mit
base_model: Aharneish/gpt2-spiritual
tags:
- generated_from_trainer
model-index:
- name: gpt-2-spiritualtest-LoRA
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# gpt-2-spiritualtest-LoRA

This model is a fine-tuned version of [Aharneish/gpt2-spiritual](https://huggingface.co/Aharneish/gpt2-spiritual) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6679

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 300

### Training results

| Training Loss | Epoch  | Step  | Validation Loss |
|:-------------:|:------:|:-----:|:---------------:|
| 2.489         | 2.12   | 500   | 1.9065          |
| 2.2722        | 4.24   | 1000  | 1.6764          |
| 2.1401        | 6.36   | 1500  | 1.5225          |
| 2.0433        | 8.47   | 2000  | 1.3953          |
| 1.9827        | 10.59  | 2500  | 1.3053          |
| 1.9249        | 12.71  | 3000  | 1.2289          |
| 1.8814        | 14.83  | 3500  | 1.1599          |
| 1.8562        | 16.95  | 4000  | 1.1164          |
| 1.8285        | 19.07  | 4500  | 1.0753          |
| 1.8037        | 21.19  | 5000  | 1.0442          |
| 1.7835        | 23.31  | 5500  | 1.0104          |
| 1.7675        | 25.42  | 6000  | 0.9916          |
| 1.7554        | 27.54  | 6500  | 0.9726          |
| 1.7389        | 29.66  | 7000  | 0.9672          |
| 1.7284        | 31.78  | 7500  | 0.9443          |
| 1.7196        | 33.9   | 8000  | 0.9335          |
| 1.7104        | 36.02  | 8500  | 0.9153          |
| 1.7013        | 38.14  | 9000  | 0.9058          |
| 1.6862        | 40.25  | 9500  | 0.8875          |
| 1.6828        | 42.37  | 10000 | 0.8942          |
| 1.6779        | 44.49  | 10500 | 0.8804          |
| 1.67          | 46.61  | 11000 | 0.8699          |
| 1.6648        | 48.73  | 11500 | 0.8617          |
| 1.6576        | 50.85  | 12000 | 0.8481          |
| 1.6506        | 52.97  | 12500 | 0.8562          |
| 1.647         | 55.08  | 13000 | 0.8444          |
| 1.6382        | 57.2   | 13500 | 0.8349          |
| 1.6401        | 59.32  | 14000 | 0.8380          |
| 1.6304        | 61.44  | 14500 | 0.8254          |
| 1.6283        | 63.56  | 15000 | 0.8234          |
| 1.6159        | 65.68  | 15500 | 0.8119          |
| 1.622         | 67.8   | 16000 | 0.8119          |
| 1.6146        | 69.92  | 16500 | 0.8091          |
| 1.6101        | 72.03  | 17000 | 0.8034          |
| 1.6049        | 74.15  | 17500 | 0.7934          |
| 1.5976        | 76.27  | 18000 | 0.7905          |
| 1.5949        | 78.39  | 18500 | 0.7883          |
| 1.5907        | 80.51  | 19000 | 0.7874          |
| 1.5952        | 82.63  | 19500 | 0.7869          |
| 1.5843        | 84.75  | 20000 | 0.7811          |
| 1.5857        | 86.86  | 20500 | 0.7793          |
| 1.5813        | 88.98  | 21000 | 0.7725          |
| 1.5753        | 91.1   | 21500 | 0.7727          |
| 1.5725        | 93.22  | 22000 | 0.7663          |
| 1.5687        | 95.34  | 22500 | 0.7643          |
| 1.5696        | 97.46  | 23000 | 0.7667          |
| 1.5605        | 99.58  | 23500 | 0.7615          |
| 1.5681        | 101.69 | 24000 | 0.7581          |
| 1.5587        | 103.81 | 24500 | 0.7563          |
| 1.5573        | 105.93 | 25000 | 0.7559          |
| 1.5532        | 108.05 | 25500 | 0.7482          |
| 1.5488        | 110.17 | 26000 | 0.7496          |
| 1.5468        | 112.29 | 26500 | 0.7440          |
| 1.5496        | 114.41 | 27000 | 0.7427          |
| 1.5471        | 116.53 | 27500 | 0.7449          |
| 1.5367        | 118.64 | 28000 | 0.7405          |
| 1.5375        | 120.76 | 28500 | 0.7368          |
| 1.5362        | 122.88 | 29000 | 0.7302          |
| 1.5347        | 125.0  | 29500 | 0.7294          |
| 1.5309        | 127.12 | 30000 | 0.7306          |
| 1.5267        | 129.24 | 30500 | 0.7240          |
| 1.5289        | 131.36 | 31000 | 0.7288          |
| 1.523         | 133.47 | 31500 | 0.7268          |
| 1.5197        | 135.59 | 32000 | 0.7200          |
| 1.5184        | 137.71 | 32500 | 0.7192          |
| 1.5188        | 139.83 | 33000 | 0.7140          |
| 1.5161        | 141.95 | 33500 | 0.7182          |
| 1.5156        | 144.07 | 34000 | 0.7136          |
| 1.5066        | 146.19 | 34500 | 0.7079          |
| 1.5063        | 148.31 | 35000 | 0.7099          |
| 1.5103        | 150.42 | 35500 | 0.7099          |
| 1.5046        | 152.54 | 36000 | 0.7059          |
| 1.503         | 154.66 | 36500 | 0.7057          |
| 1.5005        | 156.78 | 37000 | 0.7026          |
| 1.4998        | 158.9  | 37500 | 0.7014          |
| 1.4989        | 161.02 | 38000 | 0.6996          |
| 1.4931        | 163.14 | 38500 | 0.6997          |
| 1.4915        | 165.25 | 39000 | 0.6957          |
| 1.489         | 167.37 | 39500 | 0.6974          |
| 1.4906        | 169.49 | 40000 | 0.6969          |
| 1.4859        | 171.61 | 40500 | 0.6956          |
| 1.4881        | 173.73 | 41000 | 0.6921          |
| 1.4836        | 175.85 | 41500 | 0.6928          |
| 1.4818        | 177.97 | 42000 | 0.6901          |
| 1.482         | 180.08 | 42500 | 0.6912          |
| 1.4778        | 182.2  | 43000 | 0.6885          |
| 1.4763        | 184.32 | 43500 | 0.6885          |
| 1.4807        | 186.44 | 44000 | 0.6848          |
| 1.474         | 188.56 | 44500 | 0.6833          |
| 1.4712        | 190.68 | 45000 | 0.6829          |
| 1.4715        | 192.8  | 45500 | 0.6826          |
| 1.4682        | 194.92 | 46000 | 0.6831          |
| 1.4706        | 197.03 | 46500 | 0.6819          |
| 1.4674        | 199.15 | 47000 | 0.6818          |
| 1.4804        | 201.27 | 47500 | 0.6895          |
| 1.4891        | 203.39 | 48000 | 0.6905          |
| 1.4856        | 205.51 | 48500 | 0.6900          |
| 1.4826        | 207.63 | 49000 | 0.6861          |
| 1.4833        | 209.75 | 49500 | 0.6871          |
| 1.4844        | 211.86 | 50000 | 0.6865          |
| 1.4793        | 213.98 | 50500 | 0.6859          |
| 1.4805        | 216.1  | 51000 | 0.6851          |
| 1.4749        | 218.22 | 51500 | 0.6838          |
| 1.4751        | 220.34 | 52000 | 0.6826          |
| 1.4748        | 222.46 | 52500 | 0.6809          |
| 1.4738        | 224.58 | 53000 | 0.6816          |
| 1.4759        | 226.69 | 53500 | 0.6815          |
| 1.4674        | 228.81 | 54000 | 0.6792          |
| 1.472         | 230.93 | 54500 | 0.6785          |
| 1.4681        | 233.05 | 55000 | 0.6767          |
| 1.4667        | 235.17 | 55500 | 0.6762          |
| 1.4659        | 237.29 | 56000 | 0.6766          |
| 1.4697        | 239.41 | 56500 | 0.6764          |
| 1.4621        | 241.53 | 57000 | 0.6737          |
| 1.4613        | 243.64 | 57500 | 0.6745          |
| 1.4624        | 245.76 | 58000 | 0.6737          |
| 1.4593        | 247.88 | 58500 | 0.6736          |
| 1.4621        | 250.0  | 59000 | 0.6737          |
| 1.4742        | 252.12 | 59500 | 0.6798          |
| 1.4749        | 254.24 | 60000 | 0.6790          |
| 1.471         | 256.36 | 60500 | 0.6830          |
| 1.4713        | 258.47 | 61000 | 0.6820          |
| 1.4777        | 260.59 | 61500 | 0.6793          |
| 1.4738        | 262.71 | 62000 | 0.6784          |
| 1.4692        | 264.83 | 62500 | 0.6772          |
| 1.472         | 266.95 | 63000 | 0.6758          |
| 1.4707        | 269.07 | 63500 | 0.6762          |
| 1.4654        | 271.19 | 64000 | 0.6732          |
| 1.4691        | 273.31 | 64500 | 0.6746          |
| 1.4658        | 275.42 | 65000 | 0.6746          |
| 1.4648        | 277.54 | 65500 | 0.6746          |
| 1.4622        | 279.66 | 66000 | 0.6733          |
| 1.4641        | 281.78 | 66500 | 0.6708          |
| 1.4617        | 283.9  | 67000 | 0.6724          |
| 1.4605        | 286.02 | 67500 | 0.6713          |
| 1.4593        | 288.14 | 68000 | 0.6701          |
| 1.4612        | 290.25 | 68500 | 0.6699          |
| 1.4563        | 292.37 | 69000 | 0.6693          |
| 1.4555        | 294.49 | 69500 | 0.6677          |
| 1.4565        | 296.61 | 70000 | 0.6675          |
| 1.455         | 298.73 | 70500 | 0.6679          |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1
