
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# dino-base-2023_12_01-with_custom_small_head

This model is a fine-tuned version of [facebook/dinov2-base](https://huggingface.co/facebook/dinov2-base) on the multilabel_complete_dataset dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1266
- F1 Micro: 0.8318
- F1 Macro: 0.8018
- Roc Auc: 0.8960
- Accuracy: 0.5224
- Learning Rate: 0.0000

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.01
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 90

### Training results

| Training Loss | Epoch | Step  | Validation Loss | F1 Micro | F1 Macro | Roc Auc | Accuracy | Rate   |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:--------:|:-------:|:--------:|:------:|
| 0.4702        | 1.0   | 536   | 0.4545          | 0.7549   | 0.6933   | 0.8356  | 0.4298   | 0.01   |
| 0.4091        | 2.0   | 1072  | 0.3691          | 0.7832   | 0.7315   | 0.8762  | 0.4377   | 0.01   |
| 0.3999        | 3.0   | 1608  | 0.4631          | 0.7725   | 0.7214   | 0.8643  | 0.4141   | 0.01   |
| 0.3958        | 4.0   | 2144  | 0.4028          | 0.7858   | 0.7310   | 0.8765  | 0.4534   | 0.01   |
| 0.3795        | 5.0   | 2680  | 0.5129          | 0.7452   | 0.6974   | 0.8459  | 0.4230   | 0.01   |
| 0.3877        | 6.0   | 3216  | 0.5185          | 0.7386   | 0.6805   | 0.8180  | 0.4259   | 0.01   |
| 0.3658        | 7.0   | 3752  | 0.4688          | 0.7710   | 0.7034   | 0.8525  | 0.4391   | 0.01   |
| 0.373         | 8.0   | 4288  | 0.5070          | 0.7608   | 0.7020   | 0.8647  | 0.3859   | 0.01   |
| 0.2911        | 9.0   | 4824  | 0.2327          | 0.8213   | 0.7938   | 0.8885  | 0.5088   | 0.001  |
| 0.139         | 10.0  | 5360  | 0.2238          | 0.8193   | 0.7891   | 0.8972  | 0.4987   | 0.001  |
| 0.1187        | 11.0  | 5896  | 0.2095          | 0.8169   | 0.7858   | 0.8833  | 0.5084   | 0.001  |
| 0.1084        | 12.0  | 6432  | 0.1985          | 0.8209   | 0.7995   | 0.9031  | 0.4959   | 0.001  |
| 0.1038        | 13.0  | 6968  | 0.1949          | 0.8186   | 0.7914   | 0.8941  | 0.4902   | 0.001  |
| 0.0936        | 14.0  | 7504  | 0.1806          | 0.8243   | 0.7855   | 0.8947  | 0.5013   | 0.001  |
| 0.0915        | 15.0  | 8040  | 0.1799          | 0.8163   | 0.7788   | 0.8806  | 0.5113   | 0.001  |
| 0.0875        | 16.0  | 8576  | 0.1739          | 0.8196   | 0.7848   | 0.8894  | 0.5027   | 0.001  |
| 0.0848        | 17.0  | 9112  | 0.1719          | 0.8224   | 0.7894   | 0.9007  | 0.4898   | 0.001  |
| 0.0861        | 18.0  | 9648  | 0.1686          | 0.8212   | 0.7839   | 0.8904  | 0.4962   | 0.001  |
| 0.0845        | 19.0  | 10184 | 0.1659          | 0.8179   | 0.7830   | 0.8934  | 0.4941   | 0.001  |
| 0.0824        | 20.0  | 10720 | 0.1743          | 0.8105   | 0.7826   | 0.8839  | 0.4930   | 0.001  |
| 0.0834        | 21.0  | 11256 | 0.1601          | 0.8183   | 0.7905   | 0.8959  | 0.4977   | 0.001  |
| 0.0803        | 22.0  | 11792 | 0.1617          | 0.8206   | 0.7885   | 0.8985  | 0.4970   | 0.001  |
| 0.0817        | 23.0  | 12328 | 0.1586          | 0.8190   | 0.7893   | 0.8900  | 0.5038   | 0.001  |
| 0.0821        | 24.0  | 12864 | 0.1561          | 0.8203   | 0.7798   | 0.8825  | 0.5148   | 0.001  |
| 0.0795        | 25.0  | 13400 | 0.1552          | 0.8208   | 0.7897   | 0.8981  | 0.5013   | 0.001  |
| 0.0792        | 26.0  | 13936 | 0.1544          | 0.8165   | 0.7844   | 0.8853  | 0.5048   | 0.001  |
| 0.0799        | 27.0  | 14472 | 0.1509          | 0.8235   | 0.7845   | 0.8889  | 0.5105   | 0.001  |
| 0.0795        | 28.0  | 15008 | 0.1512          | 0.8208   | 0.7842   | 0.8866  | 0.5077   | 0.001  |
| 0.079         | 29.0  | 15544 | 0.1466          | 0.8222   | 0.7879   | 0.8839  | 0.5109   | 0.001  |
| 0.0803        | 30.0  | 16080 | 0.1479          | 0.8224   | 0.7874   | 0.8962  | 0.5002   | 0.001  |
| 0.0787        | 31.0  | 16616 | 0.1639          | 0.8014   | 0.7579   | 0.8601  | 0.4948   | 0.001  |
| 0.0807        | 32.0  | 17152 | 0.1468          | 0.8230   | 0.7924   | 0.8919  | 0.4966   | 0.001  |
| 0.0776        | 33.0  | 17688 | 0.1480          | 0.8220   | 0.7917   | 0.9005  | 0.4995   | 0.001  |
| 0.0802        | 34.0  | 18224 | 0.1438          | 0.8228   | 0.7907   | 0.8971  | 0.5030   | 0.001  |
| 0.0797        | 35.0  | 18760 | 0.1497          | 0.8206   | 0.7854   | 0.8899  | 0.4909   | 0.001  |
| 0.0781        | 36.0  | 19296 | 0.1407          | 0.8267   | 0.7947   | 0.8933  | 0.5109   | 0.001  |
| 0.0791        | 37.0  | 19832 | 0.1468          | 0.8219   | 0.7714   | 0.8895  | 0.5134   | 0.001  |
| 0.082         | 38.0  | 20368 | 0.1538          | 0.8105   | 0.7883   | 0.8863  | 0.4859   | 0.001  |
| 0.0781        | 39.0  | 20904 | 0.1463          | 0.8209   | 0.7858   | 0.8920  | 0.5055   | 0.001  |
| 0.0811        | 40.0  | 21440 | 0.1469          | 0.8151   | 0.7790   | 0.8880  | 0.4977   | 0.001  |
| 0.0786        | 41.0  | 21976 | 0.1518          | 0.8167   | 0.7690   | 0.8872  | 0.5052   | 0.001  |
| 0.0775        | 42.0  | 22512 | 0.1422          | 0.8260   | 0.7913   | 0.8965  | 0.5130   | 0.001  |
| 0.0641        | 43.0  | 23048 | 0.1319          | 0.8340   | 0.8001   | 0.8963  | 0.5248   | 0.0001 |
| 0.0633        | 44.0  | 23584 | 0.1313          | 0.8326   | 0.7959   | 0.8928  | 0.5298   | 0.0001 |
| 0.0627        | 45.0  | 24120 | 0.1314          | 0.8324   | 0.7994   | 0.8955  | 0.5241   | 0.0001 |
| 0.0627        | 46.0  | 24656 | 0.1308          | 0.8324   | 0.8009   | 0.8955  | 0.5234   | 0.0001 |
| 0.0619        | 47.0  | 25192 | 0.1308          | 0.8333   | 0.7996   | 0.8959  | 0.5252   | 0.0001 |
| 0.0626        | 48.0  | 25728 | 0.1310          | 0.8333   | 0.8009   | 0.8967  | 0.5198   | 0.0001 |
| 0.063         | 49.0  | 26264 | 0.1311          | 0.8328   | 0.7989   | 0.8957  | 0.5198   | 0.0001 |
| 0.0623        | 50.0  | 26800 | 0.1308          | 0.8330   | 0.7990   | 0.8962  | 0.5234   | 0.0001 |
| 0.0627        | 51.0  | 27336 | 0.1309          | 0.8329   | 0.8008   | 0.8972  | 0.5220   | 0.0001 |
| 0.0624        | 52.0  | 27872 | 0.1305          | 0.8309   | 0.7965   | 0.8909  | 0.5255   | 0.0001 |
| 0.0626        | 53.0  | 28408 | 0.1307          | 0.8313   | 0.7992   | 0.8947  | 0.5230   | 0.0001 |
| 0.0621        | 54.0  | 28944 | 0.1304          | 0.8319   | 0.7955   | 0.8964  | 0.5223   | 0.0001 |
| 0.0631        | 55.0  | 29480 | 0.1299          | 0.8328   | 0.8001   | 0.8949  | 0.5248   | 0.0001 |
| 0.063         | 56.0  | 30016 | 0.1302          | 0.8321   | 0.7989   | 0.8956  | 0.5223   | 0.0001 |
| 0.0621        | 57.0  | 30552 | 0.1304          | 0.8290   | 0.7970   | 0.8909  | 0.5230   | 0.0001 |
| 0.0623        | 58.0  | 31088 | 0.1305          | 0.8302   | 0.7978   | 0.8906  | 0.5238   | 0.0001 |
| 0.0622        | 59.0  | 31624 | 0.1307          | 0.8308   | 0.7965   | 0.8915  | 0.5238   | 0.0001 |
| 0.0627        | 60.0  | 32160 | 0.1294          | 0.8327   | 0.7998   | 0.8944  | 0.5302   | 0.0001 |
| 0.0627        | 61.0  | 32696 | 0.1303          | 0.8319   | 0.8000   | 0.8956  | 0.5241   | 0.0001 |
| 0.0626        | 62.0  | 33232 | 0.1301          | 0.8317   | 0.7982   | 0.8904  | 0.5245   | 0.0001 |
| 0.0629        | 63.0  | 33768 | 0.1297          | 0.8322   | 0.7989   | 0.8949  | 0.5248   | 0.0001 |
| 0.0617        | 64.0  | 34304 | 0.1300          | 0.8311   | 0.7982   | 0.8920  | 0.5245   | 0.0001 |
| 0.0631        | 65.0  | 34840 | 0.1292          | 0.8319   | 0.7986   | 0.8930  | 0.5245   | 0.0001 |
| 0.0619        | 66.0  | 35376 | 0.1298          | 0.8319   | 0.7982   | 0.8922  | 0.5298   | 0.0001 |
| 0.0636        | 67.0  | 35912 | 0.1298          | 0.8324   | 0.7999   | 0.8980  | 0.5245   | 0.0001 |
| 0.0627        | 68.0  | 36448 | 0.1298          | 0.8319   | 0.8006   | 0.8985  | 0.5195   | 0.0001 |
| 0.0624        | 69.0  | 36984 | 0.1293          | 0.8309   | 0.7980   | 0.8925  | 0.5259   | 0.0001 |
| 0.0625        | 70.0  | 37520 | 0.1305          | 0.8313   | 0.7967   | 0.8939  | 0.5245   | 0.0001 |
| 0.0624        | 71.0  | 38056 | 0.1303          | 0.8284   | 0.7942   | 0.8901  | 0.5166   | 0.0001 |
| 0.0618        | 72.0  | 38592 | 0.1288          | 0.8333   | 0.8010   | 0.8947  | 0.5266   | 1e-05  |
| 0.0615        | 73.0  | 39128 | 0.1288          | 0.8324   | 0.7990   | 0.8930  | 0.5291   | 1e-05  |
| 0.0602        | 74.0  | 39664 | 0.1287          | 0.8323   | 0.7989   | 0.8937  | 0.5252   | 1e-05  |
| 0.0612        | 75.0  | 40200 | 0.1286          | 0.8326   | 0.8004   | 0.8946  | 0.5263   | 1e-05  |
| 0.0611        | 76.0  | 40736 | 0.1286          | 0.8324   | 0.8001   | 0.8948  | 0.5259   | 1e-05  |
| 0.061         | 77.0  | 41272 | 0.1287          | 0.8320   | 0.7994   | 0.8937  | 0.5280   | 1e-05  |
| 0.0603        | 78.0  | 41808 | 0.1287          | 0.8323   | 0.7996   | 0.8933  | 0.5277   | 1e-05  |
| 0.0616        | 79.0  | 42344 | 0.1286          | 0.8322   | 0.7994   | 0.8936  | 0.5270   | 1e-05  |
| 0.061         | 80.0  | 42880 | 0.1286          | 0.8319   | 0.7987   | 0.8934  | 0.5280   | 1e-05  |
| 0.0607        | 81.0  | 43416 | 0.1287          | 0.8328   | 0.8003   | 0.8938  | 0.5280   | 1e-05  |
| 0.0609        | 82.0  | 43952 | 0.1288          | 0.8321   | 0.7991   | 0.8935  | 0.5288   | 1e-05  |
| 0.0611        | 83.0  | 44488 | 0.1287          | 0.8324   | 0.7994   | 0.8937  | 0.5288   | 0.0000 |
| 0.0611        | 84.0  | 45024 | 0.1286          | 0.8325   | 0.7993   | 0.8936  | 0.5284   | 0.0000 |
| 0.0608        | 85.0  | 45560 | 0.1286          | 0.8324   | 0.7992   | 0.8935  | 0.5288   | 0.0000 |
| 0.0607        | 86.0  | 46096 | 0.1286          | 0.8324   | 0.7995   | 0.8936  | 0.5284   | 0.0000 |


### Framework versions

- Transformers 4.34.1
- Pytorch 2.1.0+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1

{"language": ["eng"], "license": "apache-2.0", "tags": ["multilabel-image-classification", "multilabel", "generated_from_trainer"], "metrics": ["accuracy"], "base_model": "facebook/dinov2-base", "model-index": [{"name": "dino-base-2023_12_01-with_custom_small_head", "results": []}]}
transformers pytorch dinov2 multilabel-image-classification multilabel generated_from_trainer eng base_model:facebook/dinov2-base license:apache-2.0 endpoints_compatible region:us