whisper-small-CV-Fleurs-lg-50hrs-v1

This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0553
• Wer: 0.3658
• Cer: 0.0941

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_hf with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.8902	1.0	4291	0.9159	0.8985	0.3281
0.6801	2.0	8582	0.6461	0.8506	0.3197
0.464	3.0	12873	0.5536	0.6985	0.2535
0.3366	4.0	17164	0.5156	0.5748	0.1772
0.2397	5.0	21455	0.5143	0.8134	0.3662
0.1632	6.0	25746	0.5357	0.5707	0.1893
0.1052	7.0	30037	0.5681	0.5247	0.1621
0.0694	8.0	34328	0.5966	0.4858	0.1576
0.0518	9.0	38619	0.6355	0.4750	0.1497
0.0432	10.0	42910	0.6592	0.4203	0.1087
0.0376	11.0	47201	0.6912	0.4118	0.1034
0.03	12.0	51492	0.6968	0.4029	0.1016
0.0243	13.0	55783	0.7105	0.4015	0.1005
0.0195	14.0	60074	0.7352	0.3885	0.0975
0.0164	15.0	64365	0.7478	0.3943	0.0997
0.0143	16.0	68656	0.7653	0.3886	0.0979
0.0121	17.0	72947	0.7959	0.3894	0.0961
0.0105	18.0	77238	0.8186	0.3808	0.0985
0.0097	19.0	81529	0.8184	0.3751	0.0953
0.0088	20.0	85820	0.8283	0.3757	0.0947
0.0079	21.0	90111	0.8337	0.3803	0.0961
0.0075	22.0	94402	0.8699	0.3749	0.0951
0.0066	23.0	98693	0.8801	0.3786	0.0984
0.0062	24.0	102984	0.8804	0.3762	0.0946
0.0056	25.0	107275	0.9125	0.3806	0.0974
0.0054	26.0	111566	0.9142	0.3748	0.0982
0.0048	27.0	115857	0.9289	0.3644	0.0917
0.0048	28.0	120148	0.9187	0.3638	0.0919
0.0041	29.0	124439	0.9412	0.3673	0.0942
0.0041	30.0	128730	0.9529	0.3630	0.0911
0.0038	31.0	133021	0.9478	0.3648	0.0936
0.0037	32.0	137312	0.9369	0.3611	0.0927
0.0035	33.0	141603	0.9438	0.3563	0.0918
0.003	34.0	145894	0.9957	0.3618	0.0900
0.0031	35.0	150185	0.9849	0.3648	0.0952
0.0027	36.0	154476	1.0105	0.3641	0.0942
0.0028	37.0	158767	0.9920	0.3618	0.0919
0.0024	38.0	163058	1.0198	0.3662	0.0956
0.0025	39.0	167349	1.0161	0.3635	0.0922
0.0025	40.0	171640	1.0349	0.3645	0.0917
0.0022	41.0	175931	1.0342	0.3618	0.0906
0.0019	42.0	180222	1.0292	0.3669	0.0970
0.0019	43.0	184513	1.0440	0.3667	0.0948
0.002	44.0	188804	1.0524	0.3629	0.0947
0.0018	45.0	193095	1.0553	0.3658	0.0941

Framework versions

• Transformers 4.46.3
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.3