
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# find_marker_both_sent_train_400_eval_40_in_context_gpt2-large

This model is a fine-tuned version of [gpt2-large](https://huggingface.co/gpt2-large) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 3.6200
- Accuracy: 0.5571

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 100.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 1.7618        | 1.0   | 125   | 2.2252          | 0.5838   |
| 0.5062        | 2.0   | 250   | 2.5993          | 0.5788   |
| 0.1516        | 3.0   | 375   | 2.8070          | 0.5760   |
| 0.076         | 4.0   | 500   | 2.8840          | 0.5758   |
| 0.0568        | 5.0   | 625   | 2.9732          | 0.5751   |
| 0.0491        | 6.0   | 750   | 3.0248          | 0.5737   |
| 0.0444        | 7.0   | 875   | 3.0973          | 0.5745   |
| 0.0422        | 8.0   | 1000  | 3.0946          | 0.5748   |
| 0.0413        | 9.0   | 1125  | 3.1319          | 0.5737   |
| 0.04          | 10.0  | 1250  | 3.1391          | 0.5734   |
| 0.0396        | 11.0  | 1375  | 3.1912          | 0.5751   |
| 0.0383        | 12.0  | 1500  | 3.1988          | 0.5737   |
| 0.037         | 13.0  | 1625  | 3.1894          | 0.5744   |
| 0.0362        | 14.0  | 1750  | 3.2351          | 0.5739   |
| 0.0364        | 15.0  | 1875  | 3.2312          | 0.5735   |
| 0.0362        | 16.0  | 2000  | 3.2144          | 0.5738   |
| 0.036         | 17.0  | 2125  | 3.2431          | 0.5731   |
| 0.0348        | 18.0  | 2250  | 3.2292          | 0.5725   |
| 0.0343        | 19.0  | 2375  | 3.2963          | 0.5721   |
| 0.0346        | 20.0  | 2500  | 3.2875          | 0.5734   |
| 0.0341        | 21.0  | 2625  | 3.2421          | 0.5731   |
| 0.0356        | 22.0  | 2750  | 3.2860          | 0.5724   |
| 0.0345        | 23.0  | 2875  | 3.3301          | 0.5697   |
| 0.0337        | 24.0  | 3000  | 3.3029          | 0.5725   |
| 0.0334        | 25.0  | 3125  | 3.3797          | 0.5722   |
| 0.0333        | 26.0  | 3250  | 3.3271          | 0.5730   |
| 0.0339        | 27.0  | 3375  | 3.3109          | 0.5698   |
| 0.0334        | 28.0  | 3500  | 3.3333          | 0.5695   |
| 0.0339        | 29.0  | 3625  | 3.3320          | 0.5702   |
| 0.034         | 30.0  | 3750  | 3.2572          | 0.5690   |
| 0.0333        | 31.0  | 3875  | 3.2704          | 0.5706   |
| 0.0327        | 32.0  | 4000  | 3.3726          | 0.5689   |
| 0.0323        | 33.0  | 4125  | 3.3328          | 0.5699   |
| 0.0331        | 34.0  | 4250  | 3.3499          | 0.5676   |
| 0.0345        | 35.0  | 4375  | 3.3428          | 0.5684   |
| 0.0342        | 36.0  | 4500  | 3.3385          | 0.5678   |
| 0.0335        | 37.0  | 4625  | 3.3637          | 0.5675   |
| 0.0332        | 38.0  | 4750  | 3.3909          | 0.5688   |
| 0.0322        | 39.0  | 4875  | 3.3886          | 0.5673   |
| 0.0325        | 40.0  | 5000  | 3.4302          | 0.5689   |
| 0.0331        | 41.0  | 5125  | 3.4281          | 0.5686   |
| 0.0322        | 42.0  | 5250  | 3.4628          | 0.5675   |
| 0.0323        | 43.0  | 5375  | 3.4881          | 0.5666   |
| 0.0319        | 44.0  | 5500  | 3.4505          | 0.5672   |
| 0.0311        | 45.0  | 5625  | 3.5113          | 0.5684   |
| 0.0317        | 46.0  | 5750  | 3.4427          | 0.5675   |
| 0.0323        | 47.0  | 5875  | 3.4550          | 0.5669   |
| 0.032         | 48.0  | 6000  | 3.4382          | 0.5665   |
| 0.0318        | 49.0  | 6125  | 3.4854          | 0.5661   |
| 0.032         | 50.0  | 6250  | 3.4409          | 0.5652   |
| 0.0319        | 51.0  | 6375  | 3.4182          | 0.5662   |
| 0.0324        | 52.0  | 6500  | 3.4502          | 0.5667   |
| 0.032         | 53.0  | 6625  | 3.4940          | 0.5650   |
| 0.0316        | 54.0  | 6750  | 3.5042          | 0.5662   |
| 0.0316        | 55.0  | 6875  | 3.5134          | 0.5657   |
| 0.0311        | 56.0  | 7000  | 3.5047          | 0.5644   |
| 0.032         | 57.0  | 7125  | 3.5117          | 0.5647   |
| 0.0326        | 58.0  | 7250  | 3.4378          | 0.5648   |
| 0.0325        | 59.0  | 7375  | 3.4734          | 0.5622   |
| 0.0324        | 60.0  | 7500  | 3.4680          | 0.5634   |
| 0.0318        | 61.0  | 7625  | 3.5410          | 0.5616   |
| 0.0308        | 62.0  | 7750  | 3.6011          | 0.5633   |
| 0.0309        | 63.0  | 7875  | 3.5634          | 0.5627   |
| 0.0322        | 64.0  | 8000  | 3.4563          | 0.5608   |
| 0.0315        | 65.0  | 8125  | 3.5225          | 0.5632   |
| 0.0314        | 66.0  | 8250  | 3.5662          | 0.5627   |
| 0.0312        | 67.0  | 8375  | 3.5466          | 0.5630   |
| 0.0316        | 68.0  | 8500  | 3.5490          | 0.5626   |
| 0.031         | 69.0  | 8625  | 3.6168          | 0.5613   |
| 0.0311        | 70.0  | 8750  | 3.5900          | 0.5611   |
| 0.0322        | 71.0  | 8875  | 3.5884          | 0.5622   |
| 0.0311        | 72.0  | 9000  | 3.5161          | 0.5599   |
| 0.0311        | 73.0  | 9125  | 3.5669          | 0.5615   |
| 0.0311        | 74.0  | 9250  | 3.5836          | 0.5612   |
| 0.0314        | 75.0  | 9375  | 3.5807          | 0.5593   |
| 0.0304        | 76.0  | 9500  | 3.6734          | 0.5599   |
| 0.0313        | 77.0  | 9625  | 3.5917          | 0.5584   |
| 0.0314        | 78.0  | 9750  | 3.6313          | 0.5592   |
| 0.0311        | 79.0  | 9875  | 3.6479          | 0.5582   |
| 0.0312        | 80.0  | 10000 | 3.6653          | 0.5578   |
| 0.0313        | 81.0  | 10125 | 3.6472          | 0.5578   |
| 0.0316        | 82.0  | 10250 | 3.6846          | 0.5578   |
| 0.0322        | 83.0  | 10375 | 3.6100          | 0.5569   |
| 0.0304        | 84.0  | 10500 | 3.6680          | 0.5556   |
| 0.0303        | 85.0  | 10625 | 3.7001          | 0.5568   |
| 0.0308        | 86.0  | 10750 | 3.6609          | 0.5584   |
| 0.0304        | 87.0  | 10875 | 3.7173          | 0.5598   |
| 0.0307        | 88.0  | 11000 | 3.6079          | 0.5600   |
| 0.0305        | 89.0  | 11125 | 3.6954          | 0.5590   |
| 0.0311        | 90.0  | 11250 | 3.6578          | 0.5596   |
| 0.0307        | 91.0  | 11375 | 3.6888          | 0.5588   |
| 0.031         | 92.0  | 11500 | 3.7096          | 0.5553   |
| 0.0304        | 93.0  | 11625 | 3.6810          | 0.5576   |
| 0.0309        | 94.0  | 11750 | 3.7104          | 0.5542   |
| 0.0316        | 95.0  | 11875 | 3.6785          | 0.5584   |
| 0.0304        | 96.0  | 12000 | 3.7340          | 0.5571   |
| 0.0302        | 97.0  | 12125 | 3.7288          | 0.5581   |
| 0.0299        | 98.0  | 12250 | 3.7927          | 0.5581   |
| 0.0299        | 99.0  | 12375 | 3.7143          | 0.5564   |
| 0.031         | 100.0 | 12500 | 3.6200          | 0.5571   |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "metrics": ["accuracy"], "base_model": "gpt2-large", "model-index": [{"name": "find_marker_both_sent_train_400_eval_40_in_context_gpt2-large", "results": []}]}
transformers pytorch gpt2 text-generation generated_from_trainer base_model:gpt2-large license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us