
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# random25eof_find_passage_train50000_eval1000_rare_gpt2-large

This model is a fine-tuned version of [gpt2-large](https://huggingface.co/gpt2-large) on the tyzhu/random25eof_find_passage_train50000_eval1000_rare dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2567
- Accuracy: 0.861

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 128
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 100.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 3.2297        | 1.0   | 790   | 2.9000          | 0.3154   |
| 3.1168        | 2.0   | 1580  | 2.8902          | 0.3162   |
| 3.0979        | 3.0   | 2370  | 2.8742          | 0.3179   |
| 3.1331        | 4.0   | 3160  | 2.8582          | 0.3191   |
| 3.0373        | 5.0   | 3950  | 2.8247          | 0.3222   |
| 2.9815        | 6.0   | 4740  | 2.7791          | 0.3288   |
| 2.9294        | 7.0   | 5530  | 2.7393          | 0.3358   |
| 2.8891        | 8.0   | 6320  | 2.7001          | 0.3427   |
| 2.8347        | 9.0   | 7110  | 2.6508          | 0.3514   |
| 2.7389        | 10.0  | 7900  | 2.5695          | 0.3647   |
| 2.5974        | 11.0  | 8690  | 2.4548          | 0.3824   |
| 2.4316        | 12.0  | 9480  | 2.3369          | 0.4017   |
| 2.2691        | 13.0  | 10270 | 2.2209          | 0.4197   |
| 2.1081        | 14.0  | 11060 | 2.0811          | 0.4436   |
| 1.9473        | 15.0  | 11850 | 1.9422          | 0.4694   |
| 1.79          | 16.0  | 12640 | 1.8015          | 0.4947   |
| 1.6325        | 17.0  | 13430 | 1.6542          | 0.5239   |
| 1.4788        | 18.0  | 14220 | 1.5095          | 0.5560   |
| 1.331         | 19.0  | 15010 | 1.3519          | 0.5895   |
| 1.1927        | 20.0  | 15800 | 1.2015          | 0.6240   |
| 1.0659        | 21.0  | 16590 | 1.0446          | 0.6604   |
| 0.951         | 22.0  | 17380 | 0.9147          | 0.6923   |
| 0.8496        | 23.0  | 18170 | 0.7755          | 0.7256   |
| 0.7632        | 24.0  | 18960 | 0.6565          | 0.7588   |
| 0.6893        | 25.0  | 19750 | 0.5585          | 0.7842   |
| 0.6276        | 26.0  | 20540 | 0.4796          | 0.8057   |
| 0.5768        | 27.0  | 21330 | 0.4186          | 0.8219   |
| 0.5348        | 28.0  | 22120 | 0.3762          | 0.8332   |
| 0.5011        | 29.0  | 22910 | 0.3486          | 0.8402   |
| 0.4723        | 30.0  | 23700 | 0.3267          | 0.8456   |
| 0.4484        | 31.0  | 24490 | 0.3109          | 0.8495   |
| 0.4282        | 32.0  | 25280 | 0.2991          | 0.8528   |
| 0.411         | 33.0  | 26070 | 0.2929          | 0.8547   |
| 0.3959        | 34.0  | 26860 | 0.2845          | 0.8557   |
| 0.3838        | 35.0  | 27650 | 0.2808          | 0.8570   |
| 0.3732        | 36.0  | 28440 | 0.2777          | 0.8579   |
| 0.3641        | 37.0  | 29230 | 0.2751          | 0.8583   |
| 0.3558        | 38.0  | 30020 | 0.2719          | 0.8590   |
| 0.3484        | 39.0  | 30810 | 0.2717          | 0.8592   |
| 0.3432        | 40.0  | 31600 | 0.2704          | 0.8592   |
| 0.3373        | 41.0  | 32390 | 0.2693          | 0.8595   |
| 0.3321        | 42.0  | 33180 | 0.2682          | 0.8599   |
| 0.3279        | 43.0  | 33970 | 0.2667          | 0.8600   |
| 0.3236        | 44.0  | 34760 | 0.2662          | 0.8596   |
| 0.3199        | 45.0  | 35550 | 0.2645          | 0.8604   |
| 0.3165        | 46.0  | 36340 | 0.2644          | 0.8603   |
| 0.3139        | 47.0  | 37130 | 0.2639          | 0.8605   |
| 0.3115        | 48.0  | 37920 | 0.2637          | 0.8605   |
| 0.3092        | 49.0  | 38710 | 0.2631          | 0.8606   |
| 0.307         | 50.0  | 39500 | 0.2617          | 0.8608   |
| 0.3054        | 51.0  | 40290 | 0.2633          | 0.8608   |
| 0.3037        | 52.0  | 41080 | 0.2615          | 0.8609   |
| 0.3019        | 53.0  | 41870 | 0.2621          | 0.8608   |
| 0.3006        | 54.0  | 42660 | 0.2608          | 0.8609   |
| 0.2991        | 55.0  | 43450 | 0.2607          | 0.861    |
| 0.298         | 56.0  | 44240 | 0.2604          | 0.8611   |
| 0.2966        | 57.0  | 45030 | 0.2614          | 0.8607   |
| 0.2958        | 58.0  | 45820 | 0.2604          | 0.8611   |
| 0.2945        | 59.0  | 46610 | 0.2600          | 0.8612   |
| 0.2931        | 60.0  | 47400 | 0.2611          | 0.8608   |
| 0.2922        | 61.0  | 48190 | 0.2606          | 0.8609   |
| 0.2922        | 62.0  | 48980 | 0.2602          | 0.8612   |
| 0.2905        | 63.0  | 49770 | 0.2599          | 0.8606   |
| 0.2901        | 64.0  | 50560 | 0.2610          | 0.8609   |
| 0.2891        | 65.0  | 51350 | 0.2608          | 0.8611   |
| 0.2888        | 66.0  | 52140 | 0.2590          | 0.8613   |
| 0.2877        | 67.0  | 52930 | 0.2590          | 0.8613   |
| 0.287         | 68.0  | 53720 | 0.2588          | 0.8611   |
| 0.2863        | 69.0  | 54510 | 0.2591          | 0.8610   |
| 0.2858        | 70.0  | 55300 | 0.2598          | 0.8609   |
| 0.2853        | 71.0  | 56090 | 0.2588          | 0.8613   |
| 0.2842        | 72.0  | 56880 | 0.2591          | 0.8610   |
| 0.2842        | 73.0  | 57670 | 0.2593          | 0.8609   |
| 0.2837        | 74.0  | 58460 | 0.2580          | 0.861    |
| 0.2833        | 75.0  | 59250 | 0.2597          | 0.8609   |
| 0.2828        | 76.0  | 60040 | 0.2586          | 0.8611   |
| 0.2821        | 77.0  | 60830 | 0.2583          | 0.8609   |
| 0.2818        | 78.0  | 61620 | 0.2584          | 0.8609   |
| 0.2813        | 79.0  | 62410 | 0.2594          | 0.8613   |
| 0.2809        | 80.0  | 63200 | 0.2579          | 0.8611   |
| 0.2804        | 81.0  | 63990 | 0.2584          | 0.8611   |
| 0.2801        | 82.0  | 64780 | 0.2567          | 0.8612   |
| 0.2797        | 83.0  | 65570 | 0.2568          | 0.8612   |
| 0.2806        | 84.0  | 66360 | 0.2574          | 0.8611   |
| 0.2789        | 85.0  | 67150 | 0.2572          | 0.8609   |
| 0.2784        | 86.0  | 67940 | 0.2567          | 0.8613   |
| 0.2784        | 87.0  | 68730 | 0.2585          | 0.8609   |
| 0.278         | 88.0  | 69520 | 0.2582          | 0.8609   |
| 0.2778        | 89.0  | 70310 | 0.2581          | 0.8610   |
| 0.2772        | 90.0  | 71100 | 0.2585          | 0.8609   |
| 0.277         | 91.0  | 71890 | 0.2576          | 0.8608   |
| 0.277         | 92.0  | 72680 | 0.2571          | 0.8612   |
| 0.2767        | 93.0  | 73470 | 0.2580          | 0.8609   |
| 0.2764        | 94.0  | 74260 | 0.2574          | 0.8611   |
| 0.276         | 95.0  | 75050 | 0.2576          | 0.8613   |
| 0.2761        | 96.0  | 75840 | 0.2561          | 0.8611   |
| 0.2755        | 97.0  | 76630 | 0.2563          | 0.8611   |
| 0.2755        | 98.0  | 77420 | 0.2571          | 0.8612   |
| 0.2752        | 99.0  | 78210 | 0.2566          | 0.8610   |
| 0.2749        | 100.0 | 79000 | 0.2567          | 0.861    |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["tyzhu/random25eof_find_passage_train50000_eval1000_rare"], "metrics": ["accuracy"], "base_model": "gpt2-large", "model-index": [{"name": "random25eof_find_passage_train50000_eval1000_rare_gpt2-large", "results": [{"task": {"type": "text-generation", "name": "Causal Language Modeling"}, "dataset": {"name": "tyzhu/random25eof_find_passage_train50000_eval1000_rare", "type": "tyzhu/random25eof_find_passage_train50000_eval1000_rare"}, "metrics": [{"type": "accuracy", "value": 0.861, "name": "Accuracy"}]}]}]}
transformers pytorch gpt2 text-generation generated_from_trainer dataset:tyzhu/random25eof_find_passage_train50000_eval1000_rare base_model:gpt2-large license:mit model-index autotrain_compatible endpoints_compatible text-generation-inference region:us