w2v-bert-2.0-CV_Fleurs_AMMI_ALFFA-sw-5hrs-v1

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9564
• Wer: 0.2347
• Cer: 0.0832

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
3.3297	0.9972	175	0.7637	0.3590	0.1248
1.348	2.0	351	0.6631	0.3172	0.1132
1.0874	2.9972	526	0.6409	0.2720	0.0965
0.9039	4.0	702	0.5691	0.2759	0.1002
0.7405	4.9972	877	0.5492	0.2552	0.0905
0.6896	6.0	1053	0.6369	0.2470	0.0855
0.5831	6.9972	1228	0.5966	0.2508	0.0893
0.5089	8.0	1404	0.6115	0.2403	0.0857
0.4478	8.9972	1579	0.6523	0.2300	0.0810
0.4046	10.0	1755	0.6435	0.2459	0.0842
0.3745	10.9972	1930	0.6615	0.2336	0.0821
0.3461	12.0	2106	0.6885	0.2466	0.0850
0.3225	12.9972	2281	0.6068	0.2524	0.0871
0.278	14.0	2457	0.6808	0.2483	0.0850
0.2494	14.9972	2632	0.7234	0.2469	0.0846
0.2273	16.0	2808	0.7661	0.2414	0.0850
0.2022	16.9972	2983	0.8284	0.2451	0.0864
0.1811	18.0	3159	0.7355	0.2431	0.0855
0.1541	18.9972	3334	0.7872	0.2426	0.0860
0.1505	20.0	3510	0.7831	0.2523	0.0875
0.1373	20.9972	3685	0.8248	0.2366	0.0845
0.1213	22.0	3861	0.8190	0.2364	0.0826
0.1161	22.9972	4036	0.8505	0.2422	0.0849
0.1031	24.0	4212	0.9564	0.2347	0.0832

Framework versions

• Transformers 4.46.1
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.1