
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# random25eof_find_passage_train5000_eval1000_rare_gpt2-large

This model is a fine-tuned version of [gpt2-large](https://huggingface.co/gpt2-large) on the tyzhu/random25eof_find_passage_train5000_eval1000_rare dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2097
- Accuracy: 0.8631

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 128
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 100.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 3.6614        | 1.0   | 86   | 3.0819          | 0.3117   |
| 3.2928        | 2.0   | 172  | 3.0393          | 0.3123   |
| 3.2233        | 3.0   | 258  | 2.9447          | 0.3154   |
| 3.1701        | 4.0   | 344  | 2.9114          | 0.3155   |
| 3.1184        | 5.0   | 430  | 2.8757          | 0.3192   |
| 3.0457        | 6.0   | 516  | 2.8260          | 0.3221   |
| 2.959         | 7.0   | 602  | 2.7579          | 0.3312   |
| 2.8915        | 8.0   | 688  | 2.6973          | 0.3413   |
| 2.8286        | 9.0   | 774  | 2.6401          | 0.3506   |
| 2.7324        | 10.0  | 860  | 2.5554          | 0.3630   |
| 2.5605        | 11.0  | 946  | 2.4315          | 0.3822   |
| 2.3353        | 12.0  | 1032 | 2.2785          | 0.4075   |
| 2.0922        | 13.0  | 1118 | 2.0833          | 0.4415   |
| 1.8483        | 14.0  | 1204 | 1.8864          | 0.4749   |
| 1.6077        | 15.0  | 1290 | 1.6695          | 0.5190   |
| 1.3883        | 16.0  | 1376 | 1.4450          | 0.5671   |
| 1.1874        | 17.0  | 1462 | 1.2195          | 0.6186   |
| 1.0148        | 18.0  | 1548 | 1.0134          | 0.6662   |
| 0.8684        | 19.0  | 1634 | 0.8313          | 0.7108   |
| 0.7497        | 20.0  | 1720 | 0.6604          | 0.7549   |
| 0.6525        | 21.0  | 1806 | 0.5337          | 0.7860   |
| 0.5754        | 22.0  | 1892 | 0.4316          | 0.8132   |
| 0.5145        | 23.0  | 1978 | 0.3616          | 0.8313   |
| 0.466         | 24.0  | 2064 | 0.3113          | 0.844    |
| 0.4273        | 25.0  | 2150 | 0.2823          | 0.8495   |
| 0.3964        | 26.0  | 2236 | 0.2632          | 0.8541   |
| 0.373         | 27.0  | 2322 | 0.2529          | 0.8554   |
| 0.3542        | 28.0  | 2408 | 0.2428          | 0.8569   |
| 0.3369        | 29.0  | 2494 | 0.2393          | 0.8579   |
| 0.3232        | 30.0  | 2580 | 0.2351          | 0.8581   |
| 0.311         | 31.0  | 2666 | 0.2298          | 0.8598   |
| 0.2992        | 32.0  | 2752 | 0.2289          | 0.8601   |
| 0.2908        | 33.0  | 2838 | 0.2235          | 0.8607   |
| 0.28          | 34.0  | 2924 | 0.2201          | 0.8612   |
| 0.2713        | 35.0  | 3010 | 0.2191          | 0.8613   |
| 0.2652        | 36.0  | 3096 | 0.2176          | 0.8617   |
| 0.2619        | 37.0  | 3182 | 0.2168          | 0.8619   |
| 0.2573        | 38.0  | 3268 | 0.2163          | 0.8619   |
| 0.2542        | 39.0  | 3354 | 0.2155          | 0.8621   |
| 0.2509        | 40.0  | 3440 | 0.2153          | 0.8617   |
| 0.2474        | 41.0  | 3526 | 0.2140          | 0.8622   |
| 0.2456        | 42.0  | 3612 | 0.2128          | 0.8625   |
| 0.2426        | 43.0  | 3698 | 0.2130          | 0.8625   |
| 0.2413        | 44.0  | 3784 | 0.2142          | 0.8625   |
| 0.2397        | 45.0  | 3870 | 0.2123          | 0.8626   |
| 0.2384        | 46.0  | 3956 | 0.2133          | 0.8621   |
| 0.2376        | 47.0  | 4042 | 0.2117          | 0.8627   |
| 0.2362        | 48.0  | 4128 | 0.2106          | 0.863    |
| 0.2351        | 49.0  | 4214 | 0.2106          | 0.8628   |
| 0.2338        | 50.0  | 4300 | 0.2120          | 0.8622   |
| 0.2335        | 51.0  | 4386 | 0.2101          | 0.8628   |
| 0.2327        | 52.0  | 4472 | 0.2121          | 0.8624   |
| 0.2317        | 53.0  | 4558 | 0.2107          | 0.8629   |
| 0.2309        | 54.0  | 4644 | 0.2115          | 0.8627   |
| 0.2303        | 55.0  | 4730 | 0.2101          | 0.8631   |
| 0.2302        | 56.0  | 4816 | 0.2116          | 0.8628   |
| 0.2301        | 57.0  | 4902 | 0.2109          | 0.8630   |
| 0.2296        | 58.0  | 4988 | 0.2121          | 0.8624   |
| 0.2291        | 59.0  | 5074 | 0.2098          | 0.8629   |
| 0.2278        | 60.0  | 5160 | 0.2101          | 0.8628   |
| 0.2279        | 61.0  | 5246 | 0.2111          | 0.8627   |
| 0.228         | 62.0  | 5332 | 0.2102          | 0.8626   |
| 0.2274        | 63.0  | 5418 | 0.2105          | 0.863    |
| 0.2269        | 64.0  | 5504 | 0.2094          | 0.8630   |
| 0.226         | 65.0  | 5590 | 0.2099          | 0.8628   |
| 0.2264        | 66.0  | 5676 | 0.2098          | 0.8631   |
| 0.2257        | 67.0  | 5762 | 0.2094          | 0.8630   |
| 0.2252        | 68.0  | 5848 | 0.2097          | 0.8628   |
| 0.2254        | 69.0  | 5934 | 0.2109          | 0.863    |
| 0.225         | 70.0  | 6020 | 0.2101          | 0.8630   |
| 0.2248        | 71.0  | 6106 | 0.2094          | 0.8629   |
| 0.2244        | 72.0  | 6192 | 0.2106          | 0.8629   |
| 0.2241        | 73.0  | 6278 | 0.2083          | 0.8631   |
| 0.2235        | 74.0  | 6364 | 0.2090          | 0.8630   |
| 0.223         | 75.0  | 6450 | 0.2091          | 0.8631   |
| 0.2228        | 76.0  | 6536 | 0.2092          | 0.8631   |
| 0.2232        | 77.0  | 6622 | 0.2100          | 0.8628   |
| 0.2236        | 78.0  | 6708 | 0.2103          | 0.8629   |
| 0.2225        | 79.0  | 6794 | 0.2098          | 0.8630   |
| 0.2228        | 80.0  | 6880 | 0.2103          | 0.8631   |
| 0.2224        | 81.0  | 6966 | 0.2101          | 0.8631   |
| 0.2222        | 82.0  | 7052 | 0.2090          | 0.8631   |
| 0.2218        | 83.0  | 7138 | 0.2095          | 0.8629   |
| 0.2218        | 84.0  | 7224 | 0.2107          | 0.8630   |
| 0.2216        | 85.0  | 7310 | 0.2096          | 0.8631   |
| 0.2213        | 86.0  | 7396 | 0.2104          | 0.8631   |
| 0.2205        | 87.0  | 7482 | 0.2087          | 0.8632   |
| 0.2213        | 88.0  | 7568 | 0.2095          | 0.8631   |
| 0.2202        | 89.0  | 7654 | 0.2082          | 0.8631   |
| 0.2199        | 90.0  | 7740 | 0.2104          | 0.8631   |
| 0.2195        | 91.0  | 7826 | 0.2101          | 0.8629   |
| 0.2192        | 92.0  | 7912 | 0.2098          | 0.8632   |
| 0.2192        | 93.0  | 7998 | 0.2101          | 0.8631   |
| 0.2192        | 94.0  | 8084 | 0.2087          | 0.8631   |
| 0.2183        | 95.0  | 8170 | 0.2095          | 0.8633   |
| 0.2188        | 96.0  | 8256 | 0.2093          | 0.8631   |
| 0.2182        | 97.0  | 8342 | 0.2114          | 0.8630   |
| 0.2186        | 98.0  | 8428 | 0.2092          | 0.8630   |
| 0.2186        | 99.0  | 8514 | 0.2095          | 0.8630   |
| 0.2188        | 100.0 | 8600 | 0.2097          | 0.8631   |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["tyzhu/random25eof_find_passage_train5000_eval1000_rare"], "metrics": ["accuracy"], "base_model": "gpt2-large", "model-index": [{"name": "random25eof_find_passage_train5000_eval1000_rare_gpt2-large", "results": [{"task": {"type": "text-generation", "name": "Causal Language Modeling"}, "dataset": {"name": "tyzhu/random25eof_find_passage_train5000_eval1000_rare", "type": "tyzhu/random25eof_find_passage_train5000_eval1000_rare"}, "metrics": [{"type": "accuracy", "value": 0.8631276595744681, "name": "Accuracy"}]}]}]}
transformers pytorch gpt2 text-generation generated_from_trainer dataset:tyzhu/random25eof_find_passage_train5000_eval1000_rare base_model:gpt2-large license:mit model-index autotrain_compatible endpoints_compatible text-generation-inference region:us