
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# lmind_hotpot_train5000_eval5000_v1_docidx_gpt2-xl

This model is a fine-tuned version of [gpt2-xl](https://huggingface.co/gpt2-xl) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7708
- Accuracy: 0.8275

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 20.0

### Training results

| Training Loss | Epoch | Step  | Accuracy | Validation Loss |
|:-------------:|:-----:|:-----:|:--------:|:---------------:|
| 2.1731        | 1.0   | 1139  | 0.7726   | 2.0424          |
| 1.7483        | 2.0   | 2278  | 0.7846   | 1.6896          |
| 1.3762        | 3.0   | 3417  | 0.7959   | 1.3885          |
| 1.069         | 4.0   | 4556  | 0.8062   | 1.1036          |
| 0.8114        | 5.0   | 5695  | 0.8141   | 0.9769          |
| 0.6139        | 6.0   | 6834  | 0.8199   | 0.8487          |
| 0.4753        | 7.0   | 7973  | 0.8230   | 0.7945          |
| 0.3658        | 8.0   | 9112  | 0.8250   | 0.7536          |
| 0.2966        | 9.0   | 10251 | 0.8261   | 0.7615          |
| 0.2514        | 10.0  | 11390 | 0.8267   | 0.7641          |
| 0.2196        | 11.0  | 12529 | 0.8270   | 0.7766          |
| 0.1977        | 12.0  | 13668 | 0.8272   | 0.7625          |
| 0.1835        | 13.0  | 14807 | 0.8273   | 0.7602          |
| 0.1741        | 14.0  | 15946 | 0.8274   | 0.7733          |
| 0.1652        | 15.0  | 17085 | 0.8274   | 0.7607          |
| 0.1609        | 16.0  | 18224 | 0.8274   | 0.7803          |
| 0.1547        | 17.0  | 19363 | 0.7841   | 0.8275          |
| 0.1532        | 18.0  | 20502 | 0.7816   | 0.8275          |
| 0.1495        | 19.0  | 21641 | 0.7935   | 0.8276          |
| 0.1475        | 20.0  | 22780 | 0.7708   | 0.8275          |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "metrics": ["accuracy"], "base_model": "gpt2-xl", "model-index": [{"name": "lmind_hotpot_train5000_eval5000_v1_docidx_gpt2-xl", "results": []}]}
transformers pytorch gpt2 text-generation generated_from_trainer base_model:gpt2-xl license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us