BEiT-RHS-NDA

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.5322
• Accuracy: 0.8318

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	8	0.6851	0.5888
0.6911	2.0	16	0.6721	0.5888
0.6739	3.0	24	0.6504	0.5888
0.6595	4.0	32	0.6432	0.5888
0.646	5.0	40	0.6317	0.6822
0.646	6.0	48	0.6175	0.6916
0.6142	7.0	56	0.6270	0.6916
0.608	8.0	64	0.6618	0.6916
0.5927	9.0	72	0.5347	0.6916
0.5333	10.0	80	0.5744	0.6449
0.5333	11.0	88	0.4974	0.7477
0.4987	12.0	96	0.5970	0.6449
0.5421	13.0	104	0.5137	0.7383
0.4881	14.0	112	0.4727	0.7664
0.4408	15.0	120	0.5161	0.7664
0.4408	16.0	128	0.6732	0.6916
0.4923	17.0	136	0.6568	0.7009
0.4135	18.0	144	0.6653	0.7009
0.4308	19.0	152	0.6032	0.7196
0.3837	20.0	160	0.4492	0.8037
0.3837	21.0	168	0.4549	0.7944
0.3297	22.0	176	0.5526	0.7664
0.3264	23.0	184	0.5172	0.7944
0.3487	24.0	192	0.5105	0.7664
0.2892	25.0	200	0.4566	0.7757
0.2892	26.0	208	0.5233	0.7944
0.2505	27.0	216	0.4817	0.7944
0.2542	28.0	224	0.5035	0.8037
0.2285	29.0	232	0.5282	0.7944
0.2053	30.0	240	0.5638	0.8131
0.2053	31.0	248	0.6190	0.7570
0.2205	32.0	256	0.6142	0.7850
0.2081	33.0	264	0.5752	0.7850
0.2075	34.0	272	0.5322	0.8318
0.2286	35.0	280	0.5313	0.7944
0.2286	36.0	288	0.5189	0.8131
0.2008	37.0	296	0.5590	0.7850
0.1884	38.0	304	0.5488	0.7944
0.1819	39.0	312	0.5563	0.8037
0.1698	40.0	320	0.5679	0.7944

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0