
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# d2p_raw_default_gpt2-large

This model is a fine-tuned version of [gpt2-large](https://huggingface.co/gpt2-large) on the tyzhu/d2p_raw default dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5221
- Accuracy: 0.8458

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10.0

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 1.0742        | 0.54  | 7    | 1.3575          | 0.8043   |
| 0.3784        | 1.08  | 14   | 1.4709          | 0.8283   |
| 0.1858        | 1.62  | 21   | 1.5004          | 0.8358   |
| 0.1197        | 2.15  | 28   | 1.4656          | 0.8390   |
| 0.1102        | 2.69  | 35   | 1.4432          | 0.8379   |
| 0.0673        | 3.23  | 42   | 1.4274          | 0.8442   |
| 0.0698        | 3.77  | 49   | 1.4594          | 0.8445   |
| 0.0495        | 4.31  | 56   | 1.4863          | 0.8435   |
| 0.051         | 4.85  | 63   | 1.4810          | 0.8420   |
| 0.0458        | 5.38  | 70   | 1.4735          | 0.8434   |
| 0.0425        | 5.92  | 77   | 1.4959          | 0.8434   |
| 0.0312        | 6.46  | 84   | 1.4917          | 0.8452   |
| 0.0328        | 7.0   | 91   | 1.4898          | 0.8439   |
| 0.0295        | 7.54  | 98   | 1.5042          | 0.8439   |
| 0.0265        | 8.08  | 105  | 1.5174          | 0.8459   |
| 0.0188        | 8.62  | 112  | 1.5233          | 0.8463   |
| 0.0233        | 9.15  | 119  | 1.5231          | 0.8462   |
| 0.0198        | 9.69  | 126  | 1.5222          | 0.8458   |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1

{"license": "mit", "tags": ["generated_from_trainer"], "datasets": ["tyzhu/d2p_raw"], "metrics": ["accuracy"], "base_model": "gpt2-large", "model-index": [{"name": "d2p_raw_default_gpt2-large", "results": [{"task": {"type": "text-generation", "name": "Causal Language Modeling"}, "dataset": {"name": "tyzhu/d2p_raw default", "type": "tyzhu/d2p_raw", "args": "default"}, "metrics": [{"type": "accuracy", "value": 0.8457536557930259, "name": "Accuracy"}]}]}]}
transformers pytorch gpt2 text-generation generated_from_trainer dataset:tyzhu/d2p_raw base_model:gpt2-large license:mit model-index autotrain_compatible endpoints_compatible text-generation-inference region:us