

# gpt2-finetuned-mrpc

<!-- Provide a quick summary of what the model is/does. -->

This model is GPT-2 fine-tuned on GLUE MRPC dataset. It acheives the following results on the validation set
- Accuracy: 0.71569


## Model Details
GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. 
This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences. 
However, it acheives very good results on Text Classification tasks.
## Training procedure


### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-5
- train_batch_size: 16
- eval_batch_size: 16
- seed: 123
- optimizer: epsilon=1e-08
- num_epochs: 3

### Training results



|Epoch | Training Loss | Training Accuracy | Validation Loss | Validation Accuracy |
|:----:|:-------------:|:-----------------:|:---------------:|:-------------------:|
|   1  |     0.62922   | 0.67912           |     0.59671     |       0.69363       |
|   2  |     0.59635   | 0.69602           |     0.58504     |       0.70833       | 
|   3  |     0.57747   | 0.70311           |     0.59016     |       **0.71569**       | 

{"language": ["en"], "license": "apache-2.0", "datasets": ["glue"], "metrics": ["accuracy"], "model-index": [{"name": "gpt2-finetuned-mrpc", "results": [{"task": {"type": "text-classification", "name": "Text Classification"}, "dataset": {"name": "GLUE MRPC", "type": "glue", "args": "mrpc"}, "metrics": [{"type": "accuracy", "value": 0.71569, "name": "Accuracy"}]}]}]}
transformers pytorch gpt2 text-classification en dataset:glue license:apache-2.0 model-index autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #gpt2 #text-classification #en #dataset-glue #license-apache-2.0 #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

gpt2-finetuned-mrpc
===================


This model is GPT-2 fine-tuned on GLUE MRPC dataset. It acheives the following results on the validation set


* Accuracy: 0.71569


Model Details
-------------


GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion.
This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences.
However, it acheives very good results on Text Classification tasks.


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-5
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 123
* optimizer: epsilon=1e-08
* num\_epochs: 3


### Training results




### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-5
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 123
* optimizer: epsilon=1e-08
* num\_epochs: 3 ### Training results
TAGS
#transformers #pytorch #gpt2 #text-classification #en #dataset-glue #license-apache-2.0 #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-5
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 123
* optimizer: epsilon=1e-08
* num\_epochs: 3 ### Training results
TAGS
#transformers #pytorch #gpt2 #text-classification #en #dataset-glue #license-apache-2.0 #model-index #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-5
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 123
* optimizer: epsilon=1e-08
* num\_epochs: 3### Training results