transformers pytorch bert generated_from_trainer base_model:bert-base-uncased license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
base_model: bert-base-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: scenario-KD-SCR-DIV2-data-glue-mnli-model-bert-base-uncased-run-2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# scenario-KD-SCR-DIV2-data-glue-mnli-model-bert-base-uncased-run-2

This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2367
- Accuracy: 0.8049

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 32
- seed: 43
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 6969

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 2.6405        | 0.16  | 2000  | 2.2198          | 0.7209   |
| 2.2772        | 0.33  | 4000  | 1.9419          | 0.7557   |
| 2.026         | 0.49  | 6000  | 1.7566          | 0.7672   |
| 1.9174        | 0.65  | 8000  | 1.6522          | 0.7656   |
| 1.819         | 0.81  | 10000 | 1.5463          | 0.7801   |
| 1.7275        | 0.98  | 12000 | 1.4963          | 0.7825   |
| 1.3495        | 1.14  | 14000 | 1.5204          | 0.7815   |
| 1.3524        | 1.3   | 16000 | 1.4291          | 0.7926   |
| 1.3411        | 1.47  | 18000 | 1.5012          | 0.7856   |
| 1.3666        | 1.63  | 20000 | 1.4358          | 0.7854   |
| 1.332         | 1.79  | 22000 | 1.3944          | 0.7888   |
| 1.3553        | 1.96  | 24000 | 1.3460          | 0.7943   |
| 1.0068        | 2.12  | 26000 | 1.3837          | 0.7933   |
| 1.0256        | 2.28  | 28000 | 1.3740          | 0.7920   |
| 1.0292        | 2.44  | 30000 | 1.3959          | 0.7917   |
| 1.0785        | 2.61  | 32000 | 1.3517          | 0.7954   |
| 1.0388        | 2.77  | 34000 | 1.3471          | 0.7971   |
| 1.0582        | 2.93  | 36000 | 1.3320          | 0.7960   |
| 0.8611        | 3.1   | 38000 | 1.3254          | 0.7949   |
| 0.8928        | 3.26  | 40000 | 1.3123          | 0.8007   |
| 0.8897        | 3.42  | 42000 | 1.3417          | 0.7968   |
| 0.9109        | 3.59  | 44000 | 1.2950          | 0.8031   |
| 0.9056        | 3.75  | 46000 | 1.3281          | 0.7975   |
| 0.9068        | 3.91  | 48000 | 1.2915          | 0.8049   |
| 0.7578        | 4.07  | 50000 | 1.2832          | 0.8042   |
| 0.7766        | 4.24  | 52000 | 1.2885          | 0.7971   |
| 0.7812        | 4.4   | 54000 | 1.3066          | 0.8044   |
| 0.7979        | 4.56  | 56000 | 1.2805          | 0.8055   |
| 0.8013        | 4.73  | 58000 | 1.2759          | 0.8021   |
| 0.8196        | 4.89  | 60000 | 1.2817          | 0.7986   |
| 0.7195        | 5.05  | 62000 | 1.3076          | 0.8057   |
| 0.7154        | 5.22  | 64000 | 1.2802          | 0.8002   |
| 0.7225        | 5.38  | 66000 | 1.2483          | 0.7982   |
| 0.7492        | 5.54  | 68000 | 1.2637          | 0.8025   |
| 0.7345        | 5.7   | 70000 | 1.3132          | 0.7947   |
| 0.7505        | 5.87  | 72000 | 1.2367          | 0.8049   |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.1.1
- Datasets 2.14.5
- Tokenizers 0.13.3
