
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# lmind_nq_train600_eval300_v1_qa_gpt2-xl

This model is a fine-tuned version of [gpt2-xl](https://huggingface.co/gpt2-xl) on the tyzhu/lmind_nq_train600_eval300_v1_qa dataset.
It achieves the following results on the evaluation set:
- Loss: 3.3220
- Accuracy: 0.4313

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 20.0

### Training results

| Training Loss | Epoch | Step | Accuracy | Validation Loss |
|:-------------:|:-----:|:----:|:--------:|:---------------:|
| 2.9364        | 0.5   | 19   | 0.4152   | 2.6694          |
| 2.6902        | 1.0   | 38   | 0.4349   | 2.4354          |
| 1.8861        | 1.5   | 57   | 0.4411   | 2.4630          |
| 1.8558        | 2.0   | 76   | 0.4459   | 2.4133          |
| 1.1322        | 2.5   | 95   | 0.4435   | 2.5961          |
| 1.1703        | 3.0   | 114  | 0.4399   | 2.6530          |
| 0.7335        | 3.5   | 133  | 0.4365   | 2.8148          |
| 0.8011        | 4.0   | 152  | 0.4373   | 2.7692          |
| 0.5915        | 4.5   | 171  | 0.4328   | 2.9209          |
| 0.6364        | 5.0   | 190  | 0.4398   | 2.8734          |
| 0.5284        | 5.5   | 209  | 0.4310   | 2.8872          |
| 0.5708        | 6.0   | 228  | 0.4349   | 2.9850          |
| 0.5081        | 6.5   | 247  | 0.4317   | 2.9768          |
| 0.5024        | 7.0   | 266  | 0.4261   | 3.0177          |
| 0.4611        | 7.5   | 285  | 0.4294   | 3.0438          |
| 0.4704        | 8.0   | 304  | 0.4334   | 3.1156          |
| 0.4459        | 8.5   | 323  | 0.4348   | 3.0512          |
| 0.4853        | 9.0   | 342  | 0.4354   | 3.0102          |
| 0.4206        | 9.5   | 361  | 0.4323   | 3.1331          |
| 0.4797        | 10.0  | 380  | 0.4328   | 3.1085          |
| 0.4007        | 10.5  | 399  | 3.1363   | 0.4308          |
| 0.4113        | 11.0  | 418  | 3.1636   | 0.4318          |
| 0.4041        | 11.5  | 437  | 3.1721   | 0.4288          |
| 0.4104        | 12.0  | 456  | 3.0625   | 0.4361          |
| 0.3833        | 12.5  | 475  | 3.2416   | 0.4320          |
| 0.455         | 13.0  | 494  | 3.1030   | 0.4352          |
| 0.3727        | 13.5  | 513  | 3.2733   | 0.4309          |
| 0.4565        | 14.0  | 532  | 3.1263   | 0.4345          |
| 0.3775        | 14.5  | 551  | 3.3292   | 0.4323          |
| 0.4158        | 15.0  | 570  | 3.1972   | 0.4349          |
| 0.3868        | 15.5  | 589  | 3.3102   | 0.4298          |
| 0.3968        | 16.0  | 608  | 3.2229   | 0.4339          |
| 0.3762        | 16.5  | 627  | 3.3333   | 0.4331          |
| 0.3828        | 17.0  | 646  | 3.2512   | 0.4320          |
| 0.38          | 17.5  | 665  | 3.3682   | 0.4325          |
| 0.4261        | 18.0  | 684  | 3.2778   | 0.4275          |
| 0.3854        | 18.5  | 703  | 3.3517   | 0.4260          |
| 0.4229        | 19.0  | 722  | 3.2292   | 0.4274          |
| 0.3787        | 19.5  | 741  | 3.3292   | 0.4338          |
| 0.4098        | 20.0  | 760  | 3.3220   | 0.4313          |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["tyzhu/lmind_nq_train600_eval300_v1_qa"], "metrics": ["accuracy"], "base_model": "gpt2-xl", "model-index": [{"name": "lmind_nq_train600_eval300_v1_qa_gpt2-xl", "results": [{"task": {"type": "text-generation", "name": "Causal Language Modeling"}, "dataset": {"name": "tyzhu/lmind_nq_train600_eval300_v1_qa", "type": "tyzhu/lmind_nq_train600_eval300_v1_qa"}, "metrics": [{"type": "accuracy", "value": 0.43129032258064515, "name": "Accuracy"}]}]}]}
transformers pytorch gpt2 text-generation generated_from_trainer dataset:tyzhu/lmind_nq_train600_eval300_v1_qa base_model:gpt2-xl license:mit model-index autotrain_compatible endpoints_compatible text-generation-inference region:us