BEiT-DMAE-XDA-REVAL-80

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 1.2505
• Accuracy: 0.8043

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.5404	1.0	60	1.2860	0.4565
1.2921	2.0	120	1.1580	0.4783
0.8862	3.0	180	0.9703	0.4783
0.5719	4.0	240	0.8845	0.6957
0.3656	5.0	300	0.7957	0.7174
0.2146	6.0	360	0.8632	0.6957
0.182	7.0	420	1.0534	0.6739
0.1809	8.0	480	1.1159	0.7609
0.1687	9.0	540	0.7967	0.7609
0.1798	10.0	600	1.1140	0.6957
0.1961	11.0	660	1.6409	0.5870
0.1475	12.0	720	1.4891	0.6739
0.0964	13.0	780	1.6115	0.6739
0.0797	14.0	840	1.4461	0.6739
0.069	15.0	900	1.3023	0.6957
0.0937	16.0	960	1.2890	0.6957
0.105	17.0	1020	1.1601	0.7609
0.0909	18.0	1080	1.0927	0.7609
0.0561	19.0	1140	1.4284	0.7391
0.102	20.0	1200	1.4661	0.6739
0.0632	21.0	1260	1.2734	0.7174
0.0396	22.0	1320	1.8164	0.6522
0.0831	23.0	1380	1.5103	0.7391
0.0696	24.0	1440	1.6661	0.6739
0.0787	25.0	1500	1.5281	0.7391
0.0318	26.0	1560	1.4044	0.7609
0.056	27.0	1620	1.2505	0.8043
0.0379	28.0	1680	1.4474	0.7174
0.0475	29.0	1740	1.6855	0.6957
0.0315	30.0	1800	1.3772	0.7609
0.0661	31.0	1860	1.7190	0.6522
0.0401	32.0	1920	1.2325	0.8043
0.0503	33.0	1980	1.6231	0.7174
0.0585	34.0	2040	1.4190	0.7609
0.0338	35.0	2100	1.3640	0.7391
0.0343	36.0	2160	2.1224	0.6304
0.064	37.0	2220	2.0131	0.6739
0.0253	38.0	2280	1.8281	0.7391
0.0435	39.0	2340	1.4020	0.6957
0.0344	40.0	2400	1.4519	0.7391
0.0114	41.0	2460	1.8910	0.6957
0.0366	42.0	2520	1.3172	0.7609
0.0331	43.0	2580	1.3786	0.7391
0.0298	44.0	2640	1.3052	0.7391
0.0426	45.0	2700	1.2308	0.7826
0.0259	46.0	2760	1.6801	0.7391
0.0181	47.0	2820	1.6076	0.7174
0.0266	48.0	2880	1.4670	0.7826
0.0239	49.0	2940	1.4582	0.7174
0.0174	50.0	3000	1.4778	0.7826
0.0365	51.0	3060	1.7034	0.7174
0.0124	52.0	3120	2.0013	0.7174
0.0299	53.0	3180	1.8081	0.7174
0.0042	54.0	3240	1.6417	0.7391
0.0305	55.0	3300	1.7993	0.7391
0.026	56.0	3360	2.2341	0.7174
0.0207	57.0	3420	1.6739	0.7391
0.0063	58.0	3480	1.8008	0.7174
0.0052	59.0	3540	2.1553	0.6957
0.0061	60.0	3600	2.1446	0.6957
0.0017	61.0	3660	2.2132	0.6957
0.0143	62.0	3720	1.7295	0.7174
0.004	63.0	3780	1.5038	0.8043
0.0164	64.0	3840	1.5734	0.7609
0.0105	65.0	3900	1.6073	0.7174
0.0161	66.0	3960	1.6313	0.7391
0.0073	67.0	4020	1.7684	0.7391
0.004	68.0	4080	1.5423	0.7609
0.0096	69.0	4140	1.8458	0.7174
0.0279	70.0	4200	2.0681	0.6739
0.0067	71.0	4260	2.0441	0.6739
0.0145	72.0	4320	2.1306	0.6957
0.0084	73.0	4380	2.0877	0.6957
0.0068	74.0	4440	2.1404	0.6957
0.0044	75.0	4500	2.1604	0.6957
0.0143	76.0	4560	2.1835	0.6957
0.0105	77.0	4620	2.2572	0.6739
0.0036	78.0	4680	2.2357	0.6739
0.0205	79.0	4740	2.2578	0.6739
0.0095	80.0	4800	2.2530	0.6739

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0