swiftformer-xs-RD-da-colab

This model is a fine-tuned version of MBZUAI/swiftformer-xs on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 5.5780
• Accuracy: 0.5018

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.5414	1.0	96	9.3183	0.4945
0.533	2.0	192	7.8263	0.5
0.3449	3.0	288	8.6048	0.4945
0.3157	4.0	384	8.4227	0.4945
0.2686	5.0	480	4.3190	0.5473
0.2987	6.0	576	5.0817	0.5164
0.3415	7.0	672	5.2399	0.5127
0.2396	8.0	768	6.7857	0.5018
0.2618	9.0	864	6.5777	0.5055
0.297	10.0	960	6.6086	0.5036
0.2413	11.0	1056	3.6891	0.5236
0.2074	12.0	1152	6.8991	0.5
0.2029	13.0	1248	5.8597	0.5018
0.2353	14.0	1344	7.3848	0.5036
0.1748	15.0	1440	4.9503	0.5109
0.1885	16.0	1536	7.2151	0.4982
0.1967	17.0	1632	7.9847	0.4982
0.1881	18.0	1728	4.5008	0.5109
0.172	19.0	1824	4.7565	0.5273
0.2222	20.0	1920	6.2814	0.4964
0.1673	21.0	2016	8.1814	0.4964
0.1831	22.0	2112	4.4184	0.5164
0.1121	23.0	2208	6.0737	0.4982
0.1464	24.0	2304	5.3006	0.5018
0.1343	25.0	2400	5.6166	0.5036
0.1385	26.0	2496	6.1437	0.5018
0.1153	27.0	2592	6.3232	0.5018
0.1175	28.0	2688	6.2047	0.5036
0.1107	29.0	2784	7.5461	0.4982
0.0914	30.0	2880	7.4573	0.4982
0.1123	31.0	2976	6.2770	0.4982
0.1268	32.0	3072	5.1979	0.5073
0.1074	33.0	3168	4.9253	0.5036
0.0712	34.0	3264	5.0555	0.5018
0.0792	35.0	3360	6.1480	0.4982
0.1097	36.0	3456	6.5916	0.4982
0.1035	37.0	3552	7.4887	0.4982
0.1066	38.0	3648	6.1041	0.5
0.0887	39.0	3744	6.7739	0.4982
0.0889	40.0	3840	5.5780	0.5018

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3