
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# lmind_nq_train10000_eval6489_v1_docidx_gpt2-xl

This model is a fine-tuned version of [gpt2-xl](https://huggingface.co/gpt2-xl) on the tyzhu/lmind_nq_train10000_eval6489_v1_docidx dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5263
- Accuracy: 0.8839

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 20.0

### Training results

| Training Loss | Epoch | Step  | Accuracy | Validation Loss |
|:-------------:|:-----:|:-----:|:--------:|:---------------:|
| 2.4478        | 1.0   | 876   | 0.7054   | 2.1875          |
| 1.9693        | 2.0   | 1752  | 0.7416   | 1.7741          |
| 1.5655        | 3.0   | 2628  | 0.7823   | 1.3757          |
| 1.1895        | 4.0   | 3504  | 0.8195   | 1.0473          |
| 0.8857        | 5.0   | 4380  | 0.8470   | 0.8055          |
| 0.6375        | 6.0   | 5256  | 0.8644   | 0.6582          |
| 0.4598        | 7.0   | 6132  | 0.8740   | 0.5813          |
| 0.3373        | 8.0   | 7008  | 0.8788   | 0.5169          |
| 0.2561        | 9.0   | 7884  | 0.8808   | 0.5055          |
| 0.2083        | 10.0  | 8760  | 0.8815   | 0.5002          |
| 0.1762        | 11.0  | 9636  | 0.8826   | 0.5178          |
| 0.1581        | 12.0  | 10512 | 0.8823   | 0.5080          |
| 0.1419        | 13.0  | 11388 | 0.8826   | 0.5107          |
| 0.1307        | 14.0  | 12264 | 0.8831   | 0.5054          |
| 0.1238        | 15.0  | 13140 | 0.8828   | 0.5120          |
| 0.119         | 16.0  | 14016 | 0.8834   | 0.5269          |
| 0.1153        | 17.0  | 14892 | 0.8830   | 0.5166          |
| 0.1114        | 18.0  | 15768 | 0.4976   | 0.8833          |
| 0.1084        | 19.0  | 16644 | 0.5061   | 0.8836          |
| 0.1055        | 20.0  | 17520 | 0.5263   | 0.8839          |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["tyzhu/lmind_nq_train10000_eval6489_v1_docidx"], "metrics": ["accuracy"], "base_model": "gpt2-xl", "model-index": [{"name": "lmind_nq_train10000_eval6489_v1_docidx_gpt2-xl", "results": [{"task": {"type": "text-generation", "name": "Causal Language Modeling"}, "dataset": {"name": "tyzhu/lmind_nq_train10000_eval6489_v1_docidx", "type": "tyzhu/lmind_nq_train10000_eval6489_v1_docidx"}, "metrics": [{"type": "accuracy", "value": 0.8838953880295176, "name": "Accuracy"}]}]}]}
transformers pytorch gpt2 text-generation generated_from_trainer dataset:tyzhu/lmind_nq_train10000_eval6489_v1_docidx base_model:gpt2-xl license:mit model-index autotrain_compatible endpoints_compatible text-generation-inference region:us