transformers safetensors gpt_neox text-generation generated_from_trainer base_model:EleutherAI/pythia-70m-deduped license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: apache-2.0
base_model: EleutherAI/pythia-70m-deduped
tags:
- generated_from_trainer
model-index:
- name: chesspythia-70m-random_1M
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# chesspythia-70m-random_1M

This model is a fine-tuned version of [EleutherAI/pythia-70m-deduped](https://huggingface.co/EleutherAI/pythia-70m-deduped) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9894

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 1

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 1.4951        | 0.01  | 25   | 1.5347          |
| 1.356         | 0.02  | 50   | 1.3895          |
| 1.316         | 0.03  | 75   | 1.3312          |
| 1.2512        | 0.04  | 100  | 1.2981          |
| 1.2981        | 0.05  | 125  | 1.2738          |
| 1.2777        | 0.06  | 150  | 1.2508          |
| 1.2913        | 0.07  | 175  | 1.2362          |
| 1.2591        | 0.08  | 200  | 1.2201          |
| 1.1763        | 0.09  | 225  | 1.2136          |
| 1.1835        | 0.1   | 250  | 1.2001          |
| 1.1913        | 0.11  | 275  | 1.1864          |
| 1.1828        | 0.12  | 300  | 1.1787          |
| 1.1523        | 0.13  | 325  | 1.1708          |
| 1.2038        | 0.14  | 350  | 1.1653          |
| 1.1555        | 0.15  | 375  | 1.1591          |
| 1.1235        | 0.16  | 400  | 1.1611          |
| 1.1431        | 0.17  | 425  | 1.1484          |
| 1.1355        | 0.18  | 450  | 1.1470          |
| 1.1464        | 0.19  | 475  | 1.1354          |
| 1.146         | 0.2   | 500  | 1.1335          |
| 1.1471        | 0.21  | 525  | 1.1241          |
| 1.1568        | 0.22  | 550  | 1.1243          |
| 1.1565        | 0.23  | 575  | 1.1247          |
| 1.1253        | 0.24  | 600  | 1.1160          |
| 1.1002        | 0.25  | 625  | 1.1081          |
| 1.1356        | 0.26  | 650  | 1.1067          |
| 1.0997        | 0.27  | 675  | 1.1074          |
| 1.1092        | 0.28  | 700  | 1.1007          |
| 1.1016        | 0.29  | 725  | 1.1019          |
| 1.0578        | 0.3   | 750  | 1.0897          |
| 1.0588        | 0.31  | 775  | 1.0867          |
| 1.063         | 0.32  | 800  | 1.0901          |
| 1.0559        | 0.33  | 825  | 1.0885          |
| 1.1272        | 0.34  | 850  | 1.0841          |
| 1.0762        | 0.35  | 875  | 1.0797          |
| 1.0842        | 0.36  | 900  | 1.0793          |
| 1.0566        | 0.37  | 925  | 1.0744          |
| 1.0712        | 0.38  | 950  | 1.0682          |
| 1.076         | 0.39  | 975  | 1.0692          |
| 1.0624        | 0.4   | 1000 | 1.0670          |
| 1.0541        | 0.41  | 1025 | 1.0599          |
| 1.0292        | 0.42  | 1050 | 1.0601          |
| 1.0788        | 0.43  | 1075 | 1.0536          |
| 1.0846        | 0.44  | 1100 | 1.0530          |
| 1.0547        | 0.45  | 1125 | 1.0562          |
| 0.9978        | 0.46  | 1150 | 1.0483          |
| 1.0261        | 0.47  | 1175 | 1.0470          |
| 1.0031        | 0.48  | 1200 | 1.0475          |
| 1.0141        | 0.49  | 1225 | 1.0442          |
| 1.064         | 0.5   | 1250 | 1.0416          |
| 1.0154        | 0.51  | 1275 | 1.0408          |
| 1.0474        | 0.52  | 1300 | 1.0383          |
| 1.0392        | 0.53  | 1325 | 1.0380          |
| 1.036         | 0.54  | 1350 | 1.0360          |
| 1.0325        | 0.55  | 1375 | 1.0312          |
| 1.062         | 0.56  | 1400 | 1.0308          |
| 1.0601        | 0.57  | 1425 | 1.0282          |
| 1.0287        | 0.58  | 1450 | 1.0274          |
| 0.9944        | 0.59  | 1475 | 1.0249          |
| 1.0302        | 0.6   | 1500 | 1.0241          |
| 1.0188        | 0.61  | 1525 | 1.0210          |
| 0.8968        | 0.62  | 1550 | 1.0211          |
| 0.9661        | 0.63  | 1575 | 1.0176          |
| 1.0741        | 0.64  | 1600 | 1.0182          |
| 1.0666        | 0.65  | 1625 | 1.0156          |
| 1.0623        | 0.66  | 1650 | 1.0138          |
| 1.0435        | 0.67  | 1675 | 1.0117          |
| 0.9892        | 0.68  | 1700 | 1.0117          |
| 0.9941        | 0.69  | 1725 | 1.0097          |
| 0.967         | 0.7   | 1750 | 1.0092          |
| 1.0387        | 0.71  | 1775 | 1.0074          |
| 0.9718        | 0.72  | 1800 | 1.0056          |
| 0.9841        | 0.73  | 1825 | 1.0049          |
| 1.0086        | 0.74  | 1850 | 1.0038          |
| 0.9855        | 0.75  | 1875 | 1.0025          |
| 1.0174        | 0.76  | 1900 | 1.0022          |
| 0.9725        | 0.77  | 1925 | 1.0003          |
| 0.987         | 0.78  | 1950 | 0.9989          |
| 1.0133        | 0.79  | 1975 | 0.9979          |
| 0.9978        | 0.8   | 2000 | 0.9974          |
| 1.0193        | 0.81  | 2025 | 0.9960          |
| 0.9842        | 0.82  | 2050 | 0.9955          |
| 0.9316        | 0.83  | 2075 | 0.9952          |
| 1.0233        | 0.84  | 2100 | 0.9940          |
| 1.0054        | 0.85  | 2125 | 0.9930          |
| 0.922         | 0.86  | 2150 | 0.9933          |
| 1.0309        | 0.87  | 2175 | 0.9925          |
| 0.975         | 0.88  | 2200 | 0.9917          |
| 1.0108        | 0.89  | 2225 | 0.9915          |
| 0.9418        | 0.9   | 2250 | 0.9911          |
| 0.9445        | 0.91  | 2275 | 0.9907          |
| 0.9973        | 0.92  | 2300 | 0.9903          |
| 1.0052        | 0.93  | 2325 | 0.9900          |
| 0.9347        | 0.94  | 2350 | 0.9899          |
| 0.954         | 0.95  | 2375 | 0.9897          |
| 0.9857        | 0.96  | 2400 | 0.9896          |
| 0.958         | 0.97  | 2425 | 0.9895          |
| 1.0051        | 0.98  | 2450 | 0.9894          |
| 0.9999        | 0.99  | 2475 | 0.9894          |
| 0.959         | 1.0   | 2500 | 0.9894          |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0
