transformers safetensors vit image-classification generated_from_trainer dataset:cifar10 base_model:google/vit-base-patch16-224-in21k license:apache-2.0 model-index autotrain_compatible endpoints_compatible region:us
---
license: apache-2.0
base_model: google/vit-base-patch16-224-in21k
tags:
- generated_from_trainer
datasets:
- cifar10
metrics:
- accuracy
model-index:
- name: cifar_fine_tuning
  results:
  - task:
      name: Image Classification
      type: image-classification
    dataset:
      name: cifar10
      type: cifar10
      config: plain_text
      split: test
      args: plain_text
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.9784
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# cifar_fine_tuning

This model is a fine-tuned version of [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) on the cifar10 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1187
- Accuracy: 0.9784

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|
| 0.49          | 0.03  | 100   | 0.4099          | 0.9073   |
| 0.3651        | 0.06  | 200   | 0.2860          | 0.9267   |
| 0.3671        | 0.1   | 300   | 0.4049          | 0.8832   |
| 0.3793        | 0.13  | 400   | 0.3363          | 0.9      |
| 0.4412        | 0.16  | 500   | 0.3392          | 0.8998   |
| 0.2496        | 0.19  | 600   | 0.3285          | 0.8987   |
| 0.4256        | 0.22  | 700   | 0.3622          | 0.8919   |
| 0.4017        | 0.26  | 800   | 0.3954          | 0.8782   |
| 0.3091        | 0.29  | 900   | 0.2375          | 0.9269   |
| 0.17          | 0.32  | 1000  | 0.2150          | 0.9345   |
| 0.2855        | 0.35  | 1100  | 0.2194          | 0.9314   |
| 0.2582        | 0.38  | 1200  | 0.2473          | 0.9242   |
| 0.3127        | 0.42  | 1300  | 0.2789          | 0.9169   |
| 0.3907        | 0.45  | 1400  | 0.3239          | 0.9046   |
| 0.186         | 0.48  | 1500  | 0.2316          | 0.9315   |
| 0.2105        | 0.51  | 1600  | 0.2121          | 0.9325   |
| 0.2539        | 0.54  | 1700  | 0.2014          | 0.9376   |
| 0.3119        | 0.58  | 1800  | 0.2543          | 0.9218   |
| 0.2136        | 0.61  | 1900  | 0.2670          | 0.9183   |
| 0.2067        | 0.64  | 2000  | 0.2062          | 0.9401   |
| 0.242         | 0.67  | 2100  | 0.2852          | 0.9121   |
| 0.1743        | 0.7   | 2200  | 0.1895          | 0.9414   |
| 0.2458        | 0.74  | 2300  | 0.2358          | 0.9288   |
| 0.131         | 0.77  | 2400  | 0.2408          | 0.9221   |
| 0.4146        | 0.8   | 2500  | 0.2134          | 0.9344   |
| 0.3165        | 0.83  | 2600  | 0.2531          | 0.9248   |
| 0.2364        | 0.86  | 2700  | 0.1996          | 0.9377   |
| 0.2476        | 0.9   | 2800  | 0.1971          | 0.9393   |
| 0.298         | 0.93  | 2900  | 0.2114          | 0.9333   |
| 0.181         | 0.96  | 3000  | 0.1894          | 0.942    |
| 0.1499        | 0.99  | 3100  | 0.1819          | 0.9431   |
| 0.1636        | 1.02  | 3200  | 0.2383          | 0.9271   |
| 0.1163        | 1.06  | 3300  | 0.1888          | 0.9472   |
| 0.1179        | 1.09  | 3400  | 0.1788          | 0.9491   |
| 0.1505        | 1.12  | 3500  | 0.2084          | 0.9407   |
| 0.1135        | 1.15  | 3600  | 0.2637          | 0.9351   |
| 0.0996        | 1.18  | 3700  | 0.2281          | 0.9399   |
| 0.1469        | 1.22  | 3800  | 0.1789          | 0.9485   |
| 0.0902        | 1.25  | 3900  | 0.1599          | 0.9524   |
| 0.0456        | 1.28  | 4000  | 0.1803          | 0.9493   |
| 0.1423        | 1.31  | 4100  | 0.1510          | 0.9562   |
| 0.1269        | 1.34  | 4200  | 0.1549          | 0.9579   |
| 0.0713        | 1.38  | 4300  | 0.1833          | 0.9495   |
| 0.0731        | 1.41  | 4400  | 0.1747          | 0.9511   |
| 0.0488        | 1.44  | 4500  | 0.1530          | 0.9591   |
| 0.0538        | 1.47  | 4600  | 0.1870          | 0.9522   |
| 0.0972        | 1.5   | 4700  | 0.1547          | 0.9562   |
| 0.1294        | 1.54  | 4800  | 0.1846          | 0.9486   |
| 0.1035        | 1.57  | 4900  | 0.1609          | 0.9562   |
| 0.1564        | 1.6   | 5000  | 0.1877          | 0.9521   |
| 0.1143        | 1.63  | 5100  | 0.1606          | 0.9559   |
| 0.1239        | 1.66  | 5200  | 0.1457          | 0.9587   |
| 0.107         | 1.7   | 5300  | 0.1815          | 0.9526   |
| 0.0515        | 1.73  | 5400  | 0.1594          | 0.9568   |
| 0.0508        | 1.76  | 5500  | 0.1584          | 0.9579   |
| 0.0308        | 1.79  | 5600  | 0.1640          | 0.9543   |
| 0.0934        | 1.82  | 5700  | 0.1558          | 0.9535   |
| 0.0372        | 1.86  | 5800  | 0.1792          | 0.9481   |
| 0.063         | 1.89  | 5900  | 0.1453          | 0.9616   |
| 0.1284        | 1.92  | 6000  | 0.1719          | 0.9534   |
| 0.0987        | 1.95  | 6100  | 0.1321          | 0.9649   |
| 0.0736        | 1.98  | 6200  | 0.1754          | 0.9505   |
| 0.097         | 2.02  | 6300  | 0.1608          | 0.9565   |
| 0.0062        | 2.05  | 6400  | 0.1687          | 0.9576   |
| 0.0748        | 2.08  | 6500  | 0.2191          | 0.9488   |
| 0.0491        | 2.11  | 6600  | 0.1442          | 0.9644   |
| 0.0467        | 2.14  | 6700  | 0.1525          | 0.9636   |
| 0.042         | 2.18  | 6800  | 0.1440          | 0.9643   |
| 0.0249        | 2.21  | 6900  | 0.1526          | 0.9627   |
| 0.0887        | 2.24  | 7000  | 0.1858          | 0.9587   |
| 0.0438        | 2.27  | 7100  | 0.1485          | 0.9644   |
| 0.0434        | 2.3   | 7200  | 0.1640          | 0.9623   |
| 0.0216        | 2.34  | 7300  | 0.1685          | 0.9621   |
| 0.0496        | 2.37  | 7400  | 0.1612          | 0.9615   |
| 0.0512        | 2.4   | 7500  | 0.1554          | 0.9635   |
| 0.0173        | 2.43  | 7600  | 0.1424          | 0.9667   |
| 0.1097        | 2.46  | 7700  | 0.1691          | 0.9603   |
| 0.0072        | 2.5   | 7800  | 0.1693          | 0.9588   |
| 0.0417        | 2.53  | 7900  | 0.1669          | 0.9599   |
| 0.0624        | 2.56  | 8000  | 0.1409          | 0.9675   |
| 0.0513        | 2.59  | 8100  | 0.1401          | 0.9663   |
| 0.0083        | 2.62  | 8200  | 0.1340          | 0.9679   |
| 0.0144        | 2.66  | 8300  | 0.1378          | 0.9671   |
| 0.0958        | 2.69  | 8400  | 0.1385          | 0.9667   |
| 0.011         | 2.72  | 8500  | 0.1265          | 0.9689   |
| 0.0022        | 2.75  | 8600  | 0.1268          | 0.9671   |
| 0.0814        | 2.78  | 8700  | 0.1291          | 0.9695   |
| 0.0092        | 2.82  | 8800  | 0.1216          | 0.9714   |
| 0.0178        | 2.85  | 8900  | 0.1156          | 0.972    |
| 0.0082        | 2.88  | 9000  | 0.1070          | 0.9745   |
| 0.0325        | 2.91  | 9100  | 0.1110          | 0.9744   |
| 0.0197        | 2.94  | 9200  | 0.1244          | 0.972    |
| 0.0291        | 2.98  | 9300  | 0.1303          | 0.9719   |
| 0.0012        | 3.01  | 9400  | 0.1283          | 0.9714   |
| 0.0089        | 3.04  | 9500  | 0.1221          | 0.974    |
| 0.0115        | 3.07  | 9600  | 0.1241          | 0.9737   |
| 0.0015        | 3.1   | 9700  | 0.1335          | 0.971    |
| 0.0018        | 3.14  | 9800  | 0.1315          | 0.9716   |
| 0.0004        | 3.17  | 9900  | 0.1127          | 0.9739   |
| 0.0128        | 3.2   | 10000 | 0.1168          | 0.976    |
| 0.0194        | 3.23  | 10100 | 0.1193          | 0.9749   |
| 0.0003        | 3.26  | 10200 | 0.1145          | 0.9759   |
| 0.0056        | 3.3   | 10300 | 0.1231          | 0.9734   |
| 0.0089        | 3.33  | 10400 | 0.1392          | 0.9723   |
| 0.0032        | 3.36  | 10500 | 0.1274          | 0.9728   |
| 0.0003        | 3.39  | 10600 | 0.1236          | 0.9748   |
| 0.0002        | 3.42  | 10700 | 0.1299          | 0.9737   |
| 0.0055        | 3.46  | 10800 | 0.1307          | 0.9735   |
| 0.0008        | 3.49  | 10900 | 0.1348          | 0.9731   |
| 0.0003        | 3.52  | 11000 | 0.1345          | 0.973    |
| 0.0288        | 3.55  | 11100 | 0.1238          | 0.9759   |
| 0.0217        | 3.58  | 11200 | 0.1233          | 0.9754   |
| 0.0012        | 3.62  | 11300 | 0.1203          | 0.9768   |
| 0.0008        | 3.65  | 11400 | 0.1165          | 0.9768   |
| 0.0003        | 3.68  | 11500 | 0.1199          | 0.9776   |
| 0.0028        | 3.71  | 11600 | 0.1258          | 0.9764   |
| 0.0014        | 3.74  | 11700 | 0.1217          | 0.9766   |
| 0.0225        | 3.78  | 11800 | 0.1242          | 0.9763   |
| 0.0002        | 3.81  | 11900 | 0.1214          | 0.9778   |
| 0.0017        | 3.84  | 12000 | 0.1213          | 0.9775   |
| 0.0002        | 3.87  | 12100 | 0.1214          | 0.9778   |
| 0.001         | 3.9   | 12200 | 0.1207          | 0.9773   |
| 0.0008        | 3.94  | 12300 | 0.1190          | 0.9782   |
| 0.0002        | 3.97  | 12400 | 0.1187          | 0.9784   |
| 0.0003        | 4.0   | 12500 | 0.1187          | 0.9784   |


### Framework versions

- Transformers 4.37.0
- Pytorch 2.1.2
- Datasets 2.1.0
- Tokenizers 0.15.1
