transformers safetensors vit image-classification generated_from_trainer dataset:imagefolder base_model:google/vit-base-patch16-224 license:apache-2.0 model-index autotrain_compatible endpoints_compatible region:us
---
license: apache-2.0
base_model: google/vit-base-patch16-224
tags:
- generated_from_trainer
datasets:
- imagefolder
metrics:
- accuracy
model-index:
- name: vit-colon-cancer-classification
  results:
  - task:
      name: Image Classification
      type: image-classification
    dataset:
      name: imagefolder
      type: imagefolder
      config: default
      split: train
      args: default
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8210439105219552
pipeline_tag: image-classification
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vit-colon-cancer-classification

This model is a fine-tuned version of [google/vit-base-patch16-224](https://huggingface.co/google/vit-base-patch16-224) on the imagefolder dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6794
- Accuracy: 0.8210

## Model description

- Fine tuned vision transformer for classification of colon cancer.
- Four classes: Normal Tissue, Serrated Lesion, Adenoma, Adenocarcinoma 
  
## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 10
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 15
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.8993        | 0.35  | 100  | 0.6462          | 0.7341   |
| 0.6042        | 0.71  | 200  | 0.6380          | 0.7432   |
| 0.6284        | 1.06  | 300  | 0.5628          | 0.7821   |
| 0.5494        | 1.42  | 400  | 0.5643          | 0.7788   |
| 0.5218        | 1.77  | 500  | 0.5478          | 0.7970   |
| 0.5053        | 2.13  | 600  | 0.5356          | 0.7846   |
| 0.4441        | 2.48  | 700  | 0.6928          | 0.7133   |
| 0.4492        | 2.84  | 800  | 0.4898          | 0.8078   |
| 0.429         | 3.19  | 900  | 0.5166          | 0.8020   |
| 0.3474        | 3.55  | 1000 | 0.5373          | 0.8061   |
| 0.337         | 3.9   | 1100 | 0.5442          | 0.7904   |
| 0.3243        | 4.26  | 1200 | 0.5171          | 0.8111   |
| 0.3003        | 4.61  | 1300 | 0.5463          | 0.8070   |
| 0.3127        | 4.96  | 1400 | 0.5122          | 0.8202   |
| 0.2587        | 5.32  | 1500 | 0.5807          | 0.8152   |
| 0.2434        | 5.67  | 1600 | 0.5392          | 0.8219   |
| 0.1996        | 6.03  | 1700 | 0.6343          | 0.8045   |
| 0.2033        | 6.38  | 1800 | 0.5855          | 0.8128   |
| 0.2056        | 6.74  | 1900 | 0.6516          | 0.8144   |
| 0.1927        | 7.09  | 2000 | 0.5770          | 0.8227   |
| 0.1688        | 7.45  | 2100 | 0.6153          | 0.8293   |
| 0.1566        | 7.8   | 2200 | 0.5994          | 0.8268   |
| 0.1406        | 8.16  | 2300 | 0.6192          | 0.8277   |
| 0.1381        | 8.51  | 2400 | 0.6334          | 0.8202   |
| 0.12          | 8.87  | 2500 | 0.6444          | 0.8136   |
| 0.104         | 9.22  | 2600 | 0.6709          | 0.8202   |
| 0.1049        | 9.57  | 2700 | 0.6752          | 0.8227   |
| 0.1349        | 9.93  | 2800 | 0.6980          | 0.8186   |
| 0.0846        | 10.28 | 2900 | 0.6794          | 0.8210   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.0.1
- Datasets 2.15.0
- Tokenizers 0.15.0