
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# random25eof_find_passage_train10000_eval1000_rare_gpt2-large

This model is a fine-tuned version of [gpt2-large](https://huggingface.co/gpt2-large) on the tyzhu/random25eof_find_passage_train10000_eval1000_rare dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2201
- Accuracy: 0.8627

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 128
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 100.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 3.5009        | 1.0   | 165   | 2.9869          | 0.3135   |
| 3.2073        | 2.0   | 330   | 2.9266          | 0.3144   |
| 3.1517        | 3.0   | 495   | 2.9034          | 0.3147   |
| 3.1187        | 4.0   | 660   | 2.8829          | 0.3165   |
| 3.0713        | 5.0   | 825   | 2.8472          | 0.3199   |
| 3.0037        | 6.0   | 990   | 2.7950          | 0.3262   |
| 2.9325        | 7.0   | 1155  | 2.7399          | 0.334    |
| 2.8764        | 8.0   | 1320  | 2.6875          | 0.3438   |
| 2.8153        | 9.0   | 1485  | 2.6332          | 0.3526   |
| 2.7134        | 10.0  | 1650  | 2.5484          | 0.3663   |
| 2.5427        | 11.0  | 1815  | 2.4289          | 0.3847   |
| 2.3413        | 12.0  | 1980  | 2.2720          | 0.4090   |
| 2.1286        | 13.0  | 2145  | 2.0984          | 0.4389   |
| 1.9129        | 14.0  | 2310  | 1.9278          | 0.4696   |
| 1.6997        | 15.0  | 2475  | 1.7326          | 0.5070   |
| 1.4999        | 16.0  | 2640  | 1.5523          | 0.543    |
| 1.3096        | 17.0  | 2805  | 1.3729          | 0.5806   |
| 1.1367        | 18.0  | 2970  | 1.1750          | 0.6281   |
| 0.9831        | 19.0  | 3135  | 0.9865          | 0.6706   |
| 0.8629        | 20.0  | 3300  | 0.8282          | 0.7123   |
| 0.7508        | 21.0  | 3465  | 0.6713          | 0.7516   |
| 0.6591        | 22.0  | 3630  | 0.5575          | 0.7812   |
| 0.5858        | 23.0  | 3795  | 0.4581          | 0.8062   |
| 0.5305        | 24.0  | 3960  | 0.3880          | 0.8249   |
| 0.485         | 25.0  | 4125  | 0.3411          | 0.8371   |
| 0.4469        | 26.0  | 4290  | 0.3022          | 0.8472   |
| 0.4171        | 27.0  | 4455  | 0.2821          | 0.8508   |
| 0.3934        | 28.0  | 4620  | 0.2663          | 0.8546   |
| 0.3728        | 29.0  | 4785  | 0.2579          | 0.8562   |
| 0.3556        | 30.0  | 4950  | 0.2510          | 0.8575   |
| 0.342         | 31.0  | 5115  | 0.2469          | 0.8581   |
| 0.3308        | 32.0  | 5280  | 0.2428          | 0.8588   |
| 0.3201        | 33.0  | 5445  | 0.2409          | 0.8593   |
| 0.3107        | 34.0  | 5610  | 0.2377          | 0.8601   |
| 0.3017        | 35.0  | 5775  | 0.2361          | 0.8604   |
| 0.2944        | 36.0  | 5940  | 0.2321          | 0.8611   |
| 0.2876        | 37.0  | 6105  | 0.2309          | 0.8615   |
| 0.2815        | 38.0  | 6270  | 0.2306          | 0.8616   |
| 0.2777        | 39.0  | 6435  | 0.2303          | 0.8614   |
| 0.2739        | 40.0  | 6600  | 0.2275          | 0.862    |
| 0.2699        | 41.0  | 6765  | 0.2274          | 0.8620   |
| 0.2679        | 42.0  | 6930  | 0.2275          | 0.8619   |
| 0.2654        | 43.0  | 7095  | 0.2284          | 0.8620   |
| 0.2643        | 44.0  | 7260  | 0.2273          | 0.8619   |
| 0.262         | 45.0  | 7425  | 0.2267          | 0.8622   |
| 0.2605        | 46.0  | 7590  | 0.2256          | 0.8623   |
| 0.259         | 47.0  | 7755  | 0.2259          | 0.8621   |
| 0.2579        | 48.0  | 7920  | 0.2255          | 0.8623   |
| 0.2564        | 49.0  | 8085  | 0.2258          | 0.8620   |
| 0.2555        | 50.0  | 8250  | 0.2251          | 0.8624   |
| 0.2549        | 51.0  | 8415  | 0.2240          | 0.8625   |
| 0.2537        | 52.0  | 8580  | 0.2239          | 0.8625   |
| 0.2527        | 53.0  | 8745  | 0.2238          | 0.8624   |
| 0.2516        | 54.0  | 8910  | 0.2248          | 0.8624   |
| 0.2503        | 55.0  | 9075  | 0.2251          | 0.8620   |
| 0.2499        | 56.0  | 9240  | 0.2237          | 0.8624   |
| 0.2495        | 57.0  | 9405  | 0.2243          | 0.8623   |
| 0.2484        | 58.0  | 9570  | 0.2245          | 0.8624   |
| 0.2477        | 59.0  | 9735  | 0.2230          | 0.8626   |
| 0.2471        | 60.0  | 9900  | 0.2233          | 0.8627   |
| 0.2468        | 61.0  | 10065 | 0.2229          | 0.8627   |
| 0.2463        | 62.0  | 10230 | 0.2227          | 0.8624   |
| 0.2461        | 63.0  | 10395 | 0.2245          | 0.8625   |
| 0.2457        | 64.0  | 10560 | 0.2236          | 0.8623   |
| 0.2453        | 65.0  | 10725 | 0.2234          | 0.8625   |
| 0.2443        | 66.0  | 10890 | 0.2228          | 0.8626   |
| 0.2441        | 67.0  | 11055 | 0.2231          | 0.8625   |
| 0.2434        | 68.0  | 11220 | 0.2225          | 0.8626   |
| 0.2434        | 69.0  | 11385 | 0.2228          | 0.8626   |
| 0.2426        | 70.0  | 11550 | 0.2214          | 0.8628   |
| 0.242         | 71.0  | 11715 | 0.2222          | 0.8626   |
| 0.2418        | 72.0  | 11880 | 0.2224          | 0.8627   |
| 0.2418        | 73.0  | 12045 | 0.2217          | 0.8625   |
| 0.2416        | 74.0  | 12210 | 0.2220          | 0.8625   |
| 0.2406        | 75.0  | 12375 | 0.2214          | 0.8625   |
| 0.2404        | 76.0  | 12540 | 0.2220          | 0.8625   |
| 0.2403        | 77.0  | 12705 | 0.2217          | 0.8627   |
| 0.24          | 78.0  | 12870 | 0.2211          | 0.8626   |
| 0.2399        | 79.0  | 13035 | 0.2209          | 0.8624   |
| 0.2389        | 80.0  | 13200 | 0.2203          | 0.8627   |
| 0.239         | 81.0  | 13365 | 0.2214          | 0.8626   |
| 0.239         | 82.0  | 13530 | 0.2207          | 0.8628   |
| 0.2381        | 83.0  | 13695 | 0.2210          | 0.8626   |
| 0.2379        | 84.0  | 13860 | 0.2213          | 0.8627   |
| 0.238         | 85.0  | 14025 | 0.2217          | 0.8629   |
| 0.2376        | 86.0  | 14190 | 0.2225          | 0.8626   |
| 0.2371        | 87.0  | 14355 | 0.2225          | 0.8626   |
| 0.2377        | 88.0  | 14520 | 0.2223          | 0.8626   |
| 0.2369        | 89.0  | 14685 | 0.2222          | 0.8625   |
| 0.2364        | 90.0  | 14850 | 0.2220          | 0.8627   |
| 0.2366        | 91.0  | 15015 | 0.2206          | 0.8628   |
| 0.2361        | 92.0  | 15180 | 0.2212          | 0.8626   |
| 0.2359        | 93.0  | 15345 | 0.2211          | 0.8627   |
| 0.2361        | 94.0  | 15510 | 0.2224          | 0.8626   |
| 0.2359        | 95.0  | 15675 | 0.2210          | 0.8626   |
| 0.2354        | 96.0  | 15840 | 0.2226          | 0.8626   |
| 0.2353        | 97.0  | 16005 | 0.2211          | 0.8625   |
| 0.2351        | 98.0  | 16170 | 0.2199          | 0.8629   |
| 0.2349        | 99.0  | 16335 | 0.2204          | 0.8627   |
| 0.2345        | 100.0 | 16500 | 0.2201          | 0.8627   |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["tyzhu/random25eof_find_passage_train10000_eval1000_rare"], "metrics": ["accuracy"], "base_model": "gpt2-large", "model-index": [{"name": "random25eof_find_passage_train10000_eval1000_rare_gpt2-large", "results": [{"task": {"type": "text-generation", "name": "Causal Language Modeling"}, "dataset": {"name": "tyzhu/random25eof_find_passage_train10000_eval1000_rare", "type": "tyzhu/random25eof_find_passage_train10000_eval1000_rare"}, "metrics": [{"type": "accuracy", "value": 0.8627234042553191, "name": "Accuracy"}]}]}]}
transformers pytorch gpt2 text-generation generated_from_trainer dataset:tyzhu/random25eof_find_passage_train10000_eval1000_rare base_model:gpt2-large license:mit model-index autotrain_compatible endpoints_compatible text-generation-inference region:us