whisper-small-CV-Fleurs-lg-5hrs-v1

This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4697
• Wer: 0.5355
• Cer: 0.1316

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_hf with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
3.6431	1.0	326	1.8596	1.2267	0.5103
1.3923	2.0	652	1.2810	0.9856	0.3777
0.9641	3.0	978	1.0639	1.0332	0.4237
0.6984	4.0	1304	0.9632	1.0020	0.4011
0.4866	5.0	1630	0.9192	0.8957	0.3270
0.3137	6.0	1956	0.9370	0.8859	0.3218
0.1828	7.0	2282	0.9634	1.0643	0.4419
0.1	8.0	2608	1.0024	1.0815	0.4332
0.0623	9.0	2934	1.0291	1.0670	0.4580
0.0444	10.0	3260	1.0331	1.1076	0.4868
0.0338	11.0	3586	1.0693	0.9475	0.3759
0.0263	12.0	3912	1.0779	0.8483	0.2957
0.02	13.0	4238	1.1116	0.7602	0.2618
0.0148	14.0	4564	1.1147	0.7482	0.2480
0.0121	15.0	4890	1.1240	0.8134	0.2926
0.0093	16.0	5216	1.1651	0.6964	0.2102
0.009	17.0	5542	1.1626	0.7168	0.2301
0.0063	18.0	5868	1.1980	0.6301	0.1818
0.0055	19.0	6194	1.1901	0.6996	0.2240
0.0052	20.0	6520	1.2116	0.6431	0.1837
0.0045	21.0	6846	1.2387	0.6489	0.1873
0.0042	22.0	7172	1.2419	0.6198	0.1754
0.0035	23.0	7498	1.2544	0.6038	0.1631
0.0044	24.0	7824	1.2691	0.5742	0.1532
0.0034	25.0	8150	1.2741	0.5935	0.1536
0.0038	26.0	8476	1.2903	0.6100	0.1739
0.0031	27.0	8802	1.2731	0.5685	0.1472
0.0036	28.0	9128	1.3003	0.5696	0.1543
0.004	29.0	9454	1.2998	0.5750	0.1447
0.0025	30.0	9780	1.2927	0.5622	0.1434
0.0025	31.0	10106	1.3123	0.5661	0.1491
0.0017	32.0	10432	1.3414	0.5699	0.1488
0.003	33.0	10758	1.3225	0.5474	0.1370
0.0028	34.0	11084	1.3551	0.5568	0.1449
0.0025	35.0	11410	1.3310	0.5605	0.1413
0.0016	36.0	11736	1.3572	0.5470	0.1360
0.0028	37.0	12062	1.3139	0.5471	0.1349
0.0023	38.0	12388	1.3768	0.5459	0.1361
0.0014	39.0	12714	1.3549	0.5590	0.1449
0.0011	40.0	13040	1.3765	0.5454	0.1321
0.0026	41.0	13366	1.3808	0.5534	0.1339
0.0028	42.0	13692	1.3802	0.5484	0.1366
0.0029	43.0	14018	1.3378	0.5463	0.1350
0.0013	44.0	14344	1.3804	0.5670	0.1585
0.0016	45.0	14670	1.3573	0.5790	0.1574
0.001	46.0	14996	1.3740	0.5639	0.1585
0.0023	47.0	15322	1.3651	0.5403	0.1329
0.0011	48.0	15648	1.3826	0.5514	0.1397
0.001	49.0	15974	1.4205	0.5506	0.1345
0.0016	50.0	16300	1.4047	0.5427	0.1340
0.001	51.0	16626	1.3759	0.5502	0.1426
0.0013	52.0	16952	1.3918	0.5362	0.1332
0.0014	53.0	17278	1.4005	0.5481	0.1389
0.0009	54.0	17604	1.3985	0.5520	0.1387
0.0014	55.0	17930	1.4234	0.5490	0.1393
0.0006	56.0	18256	1.4234	0.5541	0.1478
0.0006	57.0	18582	1.4198	0.5403	0.1353
0.0003	58.0	18908	1.4268	0.5356	0.1326
0.0009	59.0	19234	1.4068	0.5477	0.1441
0.0012	60.0	19560	1.4516	0.5488	0.1402
0.0007	61.0	19886	1.4443	0.5460	0.1382
0.0001	62.0	20212	1.4444	0.5389	0.1332
0.0001	63.0	20538	1.4509	0.5420	0.1377
0.0001	64.0	20864	1.4697	0.5355	0.1316

Framework versions

• Transformers 4.46.3
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.3