
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# random25eof_find_passage_train1000_eval1000_rare_gpt2-large

This model is a fine-tuned version of [gpt2-large](https://huggingface.co/gpt2-large) on the tyzhu/random25eof_find_passage_train1000_eval1000_rare dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2144
- Accuracy: 0.8670

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 64
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 100.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 3.9638        | 1.0   | 47   | 3.1538          | 0.2911   |
| 3.4648        | 2.0   | 94   | 3.1709          | 0.3111   |
| 3.3806        | 3.0   | 141  | 3.0147          | 0.3131   |
| 3.3147        | 4.0   | 188  | 2.9886          | 0.3141   |
| 3.2712        | 5.0   | 235  | 2.9262          | 0.3172   |
| 3.1889        | 6.0   | 282  | 2.8691          | 0.3209   |
| 3.0715        | 7.0   | 329  | 2.7772          | 0.3320   |
| 2.9651        | 8.0   | 376  | 2.6855          | 0.3441   |
| 2.8663        | 9.0   | 423  | 2.6028          | 0.3567   |
| 2.7247        | 10.0  | 470  | 2.4990          | 0.3704   |
| 2.5171        | 11.0  | 517  | 2.3588          | 0.3928   |
| 2.2615        | 12.0  | 564  | 2.1660          | 0.4250   |
| 1.9813        | 13.0  | 611  | 1.9229          | 0.4680   |
| 1.7122        | 14.0  | 658  | 1.6830          | 0.5149   |
| 1.4624        | 15.0  | 705  | 1.4156          | 0.5746   |
| 1.2463        | 16.0  | 752  | 1.1552          | 0.6345   |
| 1.0564        | 17.0  | 799  | 0.9294          | 0.6918   |
| 0.9001        | 18.0  | 846  | 0.7328          | 0.7384   |
| 0.7713        | 19.0  | 893  | 0.5547          | 0.7841   |
| 0.6677        | 20.0  | 940  | 0.4444          | 0.8117   |
| 0.58          | 21.0  | 987  | 0.3570          | 0.8329   |
| 0.5057        | 22.0  | 1034 | 0.3106          | 0.8433   |
| 0.4538        | 23.0  | 1081 | 0.2822          | 0.8489   |
| 0.4076        | 24.0  | 1128 | 0.2642          | 0.8521   |
| 0.3751        | 25.0  | 1175 | 0.2473          | 0.8553   |
| 0.35          | 26.0  | 1222 | 0.2317          | 0.8597   |
| 0.3301        | 27.0  | 1269 | 0.2334          | 0.8579   |
| 0.3119        | 28.0  | 1316 | 0.2251          | 0.8607   |
| 0.2986        | 29.0  | 1363 | 0.2233          | 0.8597   |
| 0.2863        | 30.0  | 1410 | 0.2162          | 0.8618   |
| 0.2766        | 31.0  | 1457 | 0.2141          | 0.8626   |
| 0.2699        | 32.0  | 1504 | 0.2198          | 0.8605   |
| 0.2629        | 33.0  | 1551 | 0.2127          | 0.8622   |
| 0.2543        | 34.0  | 1598 | 0.2101          | 0.8629   |
| 0.2479        | 35.0  | 1645 | 0.2057          | 0.8636   |
| 0.2391        | 36.0  | 1692 | 0.2016          | 0.8649   |
| 0.2353        | 37.0  | 1739 | 0.2082          | 0.8626   |
| 0.23          | 38.0  | 1786 | 0.1997          | 0.8654   |
| 0.2265        | 39.0  | 1833 | 0.1979          | 0.8658   |
| 0.2225        | 40.0  | 1880 | 0.1983          | 0.8660   |
| 0.2183        | 41.0  | 1927 | 0.2001          | 0.8655   |
| 0.2166        | 42.0  | 1974 | 0.1950          | 0.8667   |
| 0.214         | 43.0  | 2021 | 0.1937          | 0.8668   |
| 0.211         | 44.0  | 2068 | 0.1937          | 0.8665   |
| 0.2096        | 45.0  | 2115 | 0.1970          | 0.8667   |
| 0.2096        | 46.0  | 2162 | 0.1979          | 0.8663   |
| 0.2077        | 47.0  | 2209 | 0.1950          | 0.8668   |
| 0.2061        | 48.0  | 2256 | 0.1996          | 0.8659   |
| 0.2044        | 49.0  | 2303 | 0.1984          | 0.8666   |
| 0.205         | 50.0  | 2350 | 0.1983          | 0.8667   |
| 0.2039        | 51.0  | 2397 | 0.2007          | 0.8666   |
| 0.2044        | 52.0  | 2444 | 0.1998          | 0.8667   |
| 0.2041        | 53.0  | 2491 | 0.2005          | 0.8657   |
| 0.2028        | 54.0  | 2538 | 0.2007          | 0.8667   |
| 0.2013        | 55.0  | 2585 | 0.2019          | 0.8668   |
| 0.2014        | 56.0  | 2632 | 0.2024          | 0.8666   |
| 0.1999        | 57.0  | 2679 | 0.2009          | 0.8664   |
| 0.1991        | 58.0  | 2726 | 0.2018          | 0.8663   |
| 0.1991        | 59.0  | 2773 | 0.2049          | 0.8671   |
| 0.1988        | 60.0  | 2820 | 0.2060          | 0.8671   |
| 0.1993        | 61.0  | 2867 | 0.2037          | 0.8668   |
| 0.1988        | 62.0  | 2914 | 0.2021          | 0.8668   |
| 0.1983        | 63.0  | 2961 | 0.2013          | 0.8670   |
| 0.1969        | 64.0  | 3008 | 0.2013          | 0.867    |
| 0.1963        | 65.0  | 3055 | 0.2043          | 0.8667   |
| 0.1975        | 66.0  | 3102 | 0.2042          | 0.8668   |
| 0.1964        | 67.0  | 3149 | 0.2039          | 0.8668   |
| 0.1972        | 68.0  | 3196 | 0.2062          | 0.8669   |
| 0.1964        | 69.0  | 3243 | 0.2086          | 0.8669   |
| 0.1952        | 70.0  | 3290 | 0.2056          | 0.8669   |
| 0.1951        | 71.0  | 3337 | 0.2061          | 0.8667   |
| 0.1942        | 72.0  | 3384 | 0.2053          | 0.8669   |
| 0.1942        | 73.0  | 3431 | 0.2060          | 0.8669   |
| 0.1939        | 74.0  | 3478 | 0.2065          | 0.8668   |
| 0.1934        | 75.0  | 3525 | 0.2067          | 0.8667   |
| 0.193         | 76.0  | 3572 | 0.2085          | 0.8670   |
| 0.1951        | 77.0  | 3619 | 0.2088          | 0.8666   |
| 0.1933        | 78.0  | 3666 | 0.2073          | 0.8665   |
| 0.1931        | 79.0  | 3713 | 0.2094          | 0.8670   |
| 0.1924        | 80.0  | 3760 | 0.2094          | 0.8670   |
| 0.1929        | 81.0  | 3807 | 0.2079          | 0.8669   |
| 0.1932        | 82.0  | 3854 | 0.2100          | 0.8669   |
| 0.1936        | 83.0  | 3901 | 0.2084          | 0.8663   |
| 0.1932        | 84.0  | 3948 | 0.2112          | 0.8667   |
| 0.1929        | 85.0  | 3995 | 0.2105          | 0.8671   |
| 0.1925        | 86.0  | 4042 | 0.2140          | 0.8666   |
| 0.1934        | 87.0  | 4089 | 0.2111          | 0.8670   |
| 0.1925        | 88.0  | 4136 | 0.2088          | 0.8667   |
| 0.1932        | 89.0  | 4183 | 0.2117          | 0.8667   |
| 0.1926        | 90.0  | 4230 | 0.2102          | 0.8663   |
| 0.1932        | 91.0  | 4277 | 0.2119          | 0.8667   |
| 0.1927        | 92.0  | 4324 | 0.2099          | 0.8668   |
| 0.1912        | 93.0  | 4371 | 0.2102          | 0.8665   |
| 0.1904        | 94.0  | 4418 | 0.2120          | 0.8670   |
| 0.1903        | 95.0  | 4465 | 0.2123          | 0.8669   |
| 0.1917        | 96.0  | 4512 | 0.2134          | 0.8666   |
| 0.1927        | 97.0  | 4559 | 0.2178          | 0.867    |
| 0.1922        | 98.0  | 4606 | 0.2103          | 0.8667   |
| 0.1919        | 99.0  | 4653 | 0.2114          | 0.8664   |
| 0.1918        | 100.0 | 4700 | 0.2144          | 0.8670   |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["tyzhu/random25eof_find_passage_train1000_eval1000_rare"], "metrics": ["accuracy"], "base_model": "gpt2-large", "model-index": [{"name": "random25eof_find_passage_train1000_eval1000_rare_gpt2-large", "results": [{"task": {"type": "text-generation", "name": "Causal Language Modeling"}, "dataset": {"name": "tyzhu/random25eof_find_passage_train1000_eval1000_rare", "type": "tyzhu/random25eof_find_passage_train1000_eval1000_rare"}, "metrics": [{"type": "accuracy", "value": 0.8670212765957447, "name": "Accuracy"}]}]}]}
transformers pytorch gpt2 text-generation generated_from_trainer dataset:tyzhu/random25eof_find_passage_train1000_eval1000_rare base_model:gpt2-large license:mit model-index autotrain_compatible endpoints_compatible text-generation-inference region:us