transformers pytorch bert generated_from_trainer base_model:haryoaw/scenario-TCR-data-glue-mnli-model-bert-base-uncased license:apache-2.0 endpoints_compatible region:us
---
license: apache-2.0
base_model: haryoaw/scenario-TCR-data-glue-mnli-model-bert-base-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: scenario-KD-POST-DIV6-data-glue-mnli-model-haryoaw-scenario-TCR-data-glue-mnli-m
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# scenario-KD-POST-DIV6-data-glue-mnli-model-haryoaw-scenario-TCR-data-glue-mnli-m

This model is a fine-tuned version of [haryoaw/scenario-TCR-data-glue-mnli-model-bert-base-uncased](https://huggingface.co/haryoaw/scenario-TCR-data-glue-mnli-model-bert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 2.0906
- Accuracy: 0.7414

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 6969

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 3.8471        | 0.16  | 2000  | 3.4329          | 0.6352   |
| 3.238         | 0.33  | 4000  | 3.0190          | 0.6746   |
| 3.0377        | 0.49  | 6000  | 2.7669          | 0.6857   |
| 2.8837        | 0.65  | 8000  | 2.5626          | 0.7031   |
| 2.732         | 0.81  | 10000 | 2.4540          | 0.7098   |
| 2.6409        | 0.98  | 12000 | 2.4349          | 0.7046   |
| 2.134         | 1.14  | 14000 | 2.4010          | 0.7150   |
| 2.144         | 1.3   | 16000 | 2.3651          | 0.7140   |
| 2.1536        | 1.47  | 18000 | 2.2723          | 0.7276   |
| 2.1265        | 1.63  | 20000 | 2.2793          | 0.7285   |
| 2.0902        | 1.79  | 22000 | 2.2067          | 0.7268   |
| 2.0757        | 1.96  | 24000 | 2.1944          | 0.7337   |
| 1.6915        | 2.12  | 26000 | 2.2316          | 0.7337   |
| 1.7094        | 2.28  | 28000 | 2.1919          | 0.7367   |
| 1.7199        | 2.44  | 30000 | 2.1865          | 0.7366   |
| 1.7019        | 2.61  | 32000 | 2.1866          | 0.7377   |
| 1.7082        | 2.77  | 34000 | 2.1306          | 0.7360   |
| 1.7155        | 2.93  | 36000 | 2.1693          | 0.7344   |
| 1.3872        | 3.1   | 38000 | 2.1887          | 0.7342   |
| 1.3832        | 3.26  | 40000 | 2.1632          | 0.7403   |
| 1.464         | 3.42  | 42000 | 2.1011          | 0.7369   |
| 1.4248        | 3.59  | 44000 | 2.1504          | 0.7355   |
| 1.4608        | 3.75  | 46000 | 2.1285          | 0.7388   |
| 1.4676        | 3.91  | 48000 | 2.1373          | 0.7424   |
| 1.1836        | 4.07  | 50000 | 2.1610          | 0.7383   |
| 1.1921        | 4.24  | 52000 | 2.1787          | 0.7391   |
| 1.2359        | 4.4   | 54000 | 2.1340          | 0.7409   |
| 1.2532        | 4.56  | 56000 | 2.1720          | 0.7446   |
| 1.2667        | 4.73  | 58000 | 2.1302          | 0.7386   |
| 1.271         | 4.89  | 60000 | 2.1096          | 0.7390   |
| 1.0624        | 5.05  | 62000 | 2.1759          | 0.7422   |
| 1.0852        | 5.22  | 64000 | 2.1552          | 0.7459   |
| 1.0916        | 5.38  | 66000 | 2.1637          | 0.7424   |
| 1.1108        | 5.54  | 68000 | 2.1304          | 0.7423   |
| 1.117         | 5.7   | 70000 | 2.0851          | 0.7389   |
| 1.1589        | 5.87  | 72000 | 2.1278          | 0.7435   |
| 1.035         | 6.03  | 74000 | 2.0906          | 0.7414   |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.1.1
- Datasets 2.14.5
- Tokenizers 0.13.3
