transformers tensorboard safetensors beit image-classification generated_from_trainer base_model:microsoft/beit-base-patch16-224-pt22k-ft22k license:apache-2.0 autotrain_compatible endpoints_compatible region:us
---
license: apache-2.0
base_model: microsoft/beit-base-patch16-224-pt22k-ft22k
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: finetuned-CK
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# finetuned-CK

This model is a fine-tuned version of [microsoft/beit-base-patch16-224-pt22k-ft22k](https://huggingface.co/microsoft/beit-base-patch16-224-pt22k-ft22k) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1159
- Accuracy: 0.9873

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 0.87  | 5    | 2.0523          | 0.1139   |
| 1.9841        | 1.91  | 11   | 1.9751          | 0.0633   |
| 1.9841        | 2.96  | 17   | 1.8640          | 0.1392   |
| 1.8698        | 4.0   | 23   | 1.7348          | 0.3291   |
| 1.8698        | 4.87  | 28   | 1.6262          | 0.4430   |
| 1.6947        | 5.91  | 34   | 1.4895          | 0.5823   |
| 1.6038        | 6.96  | 40   | 1.3500          | 0.5949   |
| 1.6038        | 8.0   | 46   | 1.1857          | 0.6835   |
| 1.4019        | 8.87  | 51   | 1.0636          | 0.6962   |
| 1.4019        | 9.91  | 57   | 0.9472          | 0.7468   |
| 1.2101        | 10.96 | 63   | 0.8263          | 0.7722   |
| 1.2101        | 12.0  | 69   | 0.7643          | 0.7595   |
| 1.077         | 12.87 | 74   | 0.7260          | 0.7722   |
| 0.9735        | 13.91 | 80   | 0.6628          | 0.8228   |
| 0.9735        | 14.96 | 86   | 0.6245          | 0.8101   |
| 0.8696        | 16.0  | 92   | 0.5579          | 0.8101   |
| 0.8696        | 16.87 | 97   | 0.5309          | 0.8228   |
| 0.858         | 17.91 | 103  | 0.5099          | 0.8354   |
| 0.858         | 18.96 | 109  | 0.4997          | 0.8354   |
| 0.7716        | 20.0  | 115  | 0.4679          | 0.8608   |
| 0.699         | 20.87 | 120  | 0.4455          | 0.8481   |
| 0.699         | 21.91 | 126  | 0.4409          | 0.8608   |
| 0.6768        | 22.96 | 132  | 0.4186          | 0.8481   |
| 0.6768        | 24.0  | 138  | 0.3826          | 0.8734   |
| 0.6303        | 24.87 | 143  | 0.3708          | 0.8861   |
| 0.6303        | 25.91 | 149  | 0.3545          | 0.8608   |
| 0.5786        | 26.96 | 155  | 0.3480          | 0.8987   |
| 0.5188        | 28.0  | 161  | 0.3241          | 0.9241   |
| 0.5188        | 28.87 | 166  | 0.3257          | 0.9114   |
| 0.5349        | 29.91 | 172  | 0.2963          | 0.9241   |
| 0.5349        | 30.96 | 178  | 0.2836          | 0.9367   |
| 0.5208        | 32.0  | 184  | 0.2822          | 0.9620   |
| 0.5208        | 32.87 | 189  | 0.2933          | 0.9494   |
| 0.4458        | 33.91 | 195  | 0.2742          | 0.9620   |
| 0.4716        | 34.96 | 201  | 0.2580          | 0.9620   |
| 0.4716        | 36.0  | 207  | 0.2432          | 0.9620   |
| 0.427         | 36.87 | 212  | 0.2333          | 0.9747   |
| 0.427         | 37.91 | 218  | 0.2115          | 0.9494   |
| 0.4052        | 38.96 | 224  | 0.2044          | 0.9873   |
| 0.3899        | 40.0  | 230  | 0.2081          | 0.9747   |
| 0.3899        | 40.87 | 235  | 0.2110          | 0.9620   |
| 0.4098        | 41.91 | 241  | 0.2099          | 0.9494   |
| 0.4098        | 42.96 | 247  | 0.1899          | 0.9747   |
| 0.3691        | 44.0  | 253  | 0.1783          | 0.9873   |
| 0.3691        | 44.87 | 258  | 0.1768          | 0.9747   |
| 0.3878        | 45.91 | 264  | 0.1836          | 0.9873   |
| 0.3582        | 46.96 | 270  | 0.1876          | 0.9747   |
| 0.3582        | 48.0  | 276  | 0.1761          | 0.9747   |
| 0.356         | 48.87 | 281  | 0.1745          | 0.9620   |
| 0.356         | 49.91 | 287  | 0.1761          | 0.9620   |
| 0.3715        | 50.96 | 293  | 0.1716          | 0.9873   |
| 0.3715        | 52.0  | 299  | 0.1694          | 0.9620   |
| 0.3195        | 52.87 | 304  | 0.1660          | 0.9873   |
| 0.3452        | 53.91 | 310  | 0.1584          | 0.9873   |
| 0.3452        | 54.96 | 316  | 0.1579          | 0.9620   |
| 0.3407        | 56.0  | 322  | 0.1431          | 0.9873   |
| 0.3407        | 56.87 | 327  | 0.1423          | 0.9873   |
| 0.3092        | 57.91 | 333  | 0.1434          | 0.9747   |
| 0.3092        | 58.96 | 339  | 0.1391          | 0.9873   |
| 0.3346        | 60.0  | 345  | 0.1362          | 0.9873   |
| 0.3107        | 60.87 | 350  | 0.1341          | 0.9873   |
| 0.3107        | 61.91 | 356  | 0.1368          | 0.9873   |
| 0.2884        | 62.96 | 362  | 0.1393          | 0.9747   |
| 0.2884        | 64.0  | 368  | 0.1347          | 0.9873   |
| 0.3048        | 64.87 | 373  | 0.1334          | 0.9873   |
| 0.3048        | 65.91 | 379  | 0.1342          | 0.9747   |
| 0.3452        | 66.96 | 385  | 0.1310          | 0.9747   |
| 0.2835        | 68.0  | 391  | 0.1308          | 0.9873   |
| 0.2835        | 68.87 | 396  | 0.1304          | 0.9747   |
| 0.3865        | 69.91 | 402  | 0.1264          | 0.9873   |
| 0.3865        | 70.96 | 408  | 0.1247          | 0.9873   |
| 0.2729        | 72.0  | 414  | 0.1239          | 0.9873   |
| 0.2729        | 72.87 | 419  | 0.1249          | 0.9747   |
| 0.2643        | 73.91 | 425  | 0.1189          | 0.9873   |
| 0.3176        | 74.96 | 431  | 0.1171          | 0.9873   |
| 0.3176        | 76.0  | 437  | 0.1174          | 0.9873   |
| 0.3184        | 76.87 | 442  | 0.1184          | 0.9873   |
| 0.3184        | 77.91 | 448  | 0.1160          | 0.9873   |
| 0.2817        | 78.96 | 454  | 0.1147          | 0.9873   |
| 0.2666        | 80.0  | 460  | 0.1133          | 0.9873   |
| 0.2666        | 80.87 | 465  | 0.1139          | 0.9873   |
| 0.2589        | 81.91 | 471  | 0.1142          | 0.9873   |
| 0.2589        | 82.96 | 477  | 0.1150          | 0.9873   |
| 0.275         | 84.0  | 483  | 0.1156          | 0.9873   |
| 0.275         | 84.87 | 488  | 0.1160          | 0.9873   |
| 0.2644        | 85.91 | 494  | 0.1160          | 0.9873   |
| 0.3187        | 86.96 | 500  | 0.1159          | 0.9873   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0
