whisper-small-CV-Fleurs-lg-10hrs-v1

This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2621
• Wer: 0.5324
• Cer: 0.1576

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_hf with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
3.0427	1.0	646	1.5425	1.0511	0.4113
1.1757	2.0	1292	1.0746	0.9446	0.3445
0.8236	3.0	1938	0.9023	0.8557	0.2892
0.5996	4.0	2584	0.8086	0.9530	0.3737
0.4218	5.0	3230	0.7879	0.9619	0.3807
0.2747	6.0	3876	0.8049	1.2464	0.5660
0.1645	7.0	4522	0.8353	1.0604	0.4656
0.0963	8.0	5168	0.8643	0.9788	0.4183
0.0596	9.0	5814	0.9014	0.9998	0.4201
0.0428	10.0	6460	0.9473	0.9952	0.4247
0.0349	11.0	7106	0.9611	0.9451	0.3851
0.027	12.0	7752	0.9740	0.7178	0.2365
0.0209	13.0	8398	0.9891	0.6217	0.1888
0.0163	14.0	9044	0.9843	0.5607	0.1508
0.0134	15.0	9690	1.0090	0.5533	0.1516
0.0123	16.0	10336	1.0624	0.5635	0.1528
0.0105	17.0	10982	1.0681	0.5307	0.1328
0.0089	18.0	11628	1.0781	0.5255	0.1292
0.0071	19.0	12274	1.1051	0.5288	0.1284
0.0064	20.0	12920	1.1033	0.5126	0.1319
0.0057	21.0	13566	1.1006	0.5134	0.1261
0.0051	22.0	14212	1.1209	0.5076	0.1226
0.0041	23.0	14858	1.1327	0.5198	0.1376
0.0051	24.0	15504	1.1404	0.5120	0.1249
0.0047	25.0	16150	1.1593	0.5137	0.1294
0.0041	26.0	16796	1.1772	0.5022	0.1227
0.0044	27.0	17442	1.1582	0.5033	0.1229
0.0042	28.0	18088	1.1964	0.5045	0.1232
0.0034	29.0	18734	1.2126	0.4958	0.1209
0.0034	30.0	19380	1.1914	0.4926	0.1218
0.0026	31.0	20026	1.2171	0.5254	0.1403
0.0026	32.0	20672	1.2150	0.5026	0.1276
0.003	33.0	21318	1.2793	0.4969	0.1225
0.0026	34.0	21964	1.2307	0.4961	0.1235
0.0022	35.0	22610	1.2437	0.5014	0.1233
0.0027	36.0	23256	1.2565	0.5038	0.1259
0.0025	37.0	23902	1.2610	0.5003	0.1254
0.0022	38.0	24548	1.2506	0.5032	0.1306
0.0024	39.0	25194	1.2798	0.5235	0.1492
0.0024	40.0	25840	1.2703	0.5255	0.1471
0.0017	41.0	26486	1.2574	0.5051	0.1299
0.0021	42.0	27132	1.2621	0.5324	0.1576

Framework versions

• Transformers 4.46.3
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.3