whisper-small-CV-Fleurs-lg-200hrs-v1

This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9685
• Wer: 0.2942
• Cer: 0.0795

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_hf with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.1937	1.0	19207	0.6376	0.7748	0.2786
0.3771	2.0	38414	0.4722	0.8076	0.3182
0.2553	3.0	57621	0.4182	0.7767	0.3623
0.1851	4.0	76828	0.4079	0.4659	0.1390
0.1335	5.0	96035	0.4137	0.3769	0.0976
0.0944	6.0	115242	0.4493	0.3737	0.0987
0.0675	7.0	134449	0.4601	0.3512	0.0885
0.0514	8.0	153656	0.4979	0.3439	0.0878
0.0432	9.0	172863	0.5161	0.3513	0.0920
0.0393	10.0	192070	0.5299	0.3427	0.0879
0.035	11.0	211277	0.5425	0.3308	0.0846
0.0275	12.0	230484	0.5871	0.3336	0.0897
0.0225	13.0	249691	0.5861	0.3217	0.0822
0.0188	14.0	268898	0.6070	0.3207	0.0832
0.0161	15.0	288105	0.6223	0.3220	0.0848
0.014	16.0	307312	0.6258	0.3244	0.0885
0.0121	17.0	326519	0.6718	0.3213	0.0827
0.0107	18.0	345726	0.6716	0.3235	0.0888
0.0097	19.0	364933	0.6679	0.3347	0.0994
0.0087	20.0	384140	0.6787	0.3219	0.0875
0.0081	21.0	403347	0.6896	0.3099	0.0847
0.0073	22.0	422554	0.7138	0.3156	0.0862
0.0068	23.0	441761	0.7167	0.3164	0.0858
0.0063	24.0	460968	0.7188	0.3119	0.0847
0.0058	25.0	480175	0.7432	0.3114	0.0857
0.0053	26.0	499382	0.7469	0.3203	0.0872
0.005	27.0	518589	0.7557	0.3091	0.0842
0.0048	28.0	537796	0.7797	0.3113	0.0864
0.0043	29.0	557003	0.7660	0.3064	0.0840
0.0041	30.0	576210	0.7889	0.3086	0.0835
0.0038	31.0	595417	0.8019	0.3066	0.0813
0.0036	32.0	614624	0.7947	0.3110	0.0848
0.0034	33.0	633831	0.8009	0.3046	0.0806
0.0031	34.0	653038	0.8163	0.3036	0.0829
0.0031	35.0	672245	0.8172	0.3088	0.0833
0.003	36.0	691452	0.8206	0.3121	0.0864
0.0028	37.0	710659	0.8349	0.3074	0.0854
0.0026	38.0	729866	0.8193	0.3023	0.0813
0.0026	39.0	749073	0.8335	0.3084	0.0848
0.0024	40.0	768280	0.8445	0.3023	0.0825
0.0022	41.0	787487	0.8561	0.3045	0.0837
0.0022	42.0	806694	0.8287	0.3094	0.0859
0.002	43.0	825901	0.8430	0.3035	0.0839
0.002	44.0	845108	0.8517	0.2980	0.0809
0.0017	45.0	864315	0.8633	0.3028	0.0838
0.0017	46.0	883522	0.8802	0.3055	0.0843
0.0016	47.0	902729	0.8762	0.2988	0.0804
0.0016	48.0	921936	0.8995	0.2978	0.0790
0.0014	49.0	941143	0.8810	0.3048	0.0824
0.0014	50.0	960350	0.8949	0.3019	0.0815
0.0012	51.0	979557	0.8912	0.3022	0.0832
0.0012	52.0	998764	0.8964	0.2943	0.0806
0.0012	53.0	1017971	0.8901	0.2998	0.0826
0.0011	54.0	1037178	0.8779	0.3013	0.0830
0.001	55.0	1056385	0.8977	0.3011	0.0827
0.001	56.0	1075592	0.9069	0.2948	0.0834
0.0008	57.0	1094799	0.8910	0.2965	0.0800
0.0008	58.0	1114006	0.9191	0.2954	0.0797
0.0008	59.0	1133213	0.9032	0.2889	0.0783
0.0008	60.0	1152420	0.9169	0.3005	0.0820
0.0007	61.0	1171627	0.9268	0.2970	0.0793
0.0006	62.0	1190834	0.9222	0.2935	0.0785
0.0007	63.0	1210041	0.9338	0.2969	0.0800
0.0006	64.0	1229248	0.9288	0.2988	0.0837
0.0005	65.0	1248455	0.9336	0.2934	0.0803
0.0005	66.0	1267662	0.9308	0.2974	0.0821
0.0005	67.0	1286869	0.9479	0.2895	0.0785
0.0005	68.0	1306076	0.9484	0.2988	0.0824
0.0004	69.0	1325283	0.9540	0.2916	0.0815
0.0004	70.0	1344490	0.9641	0.2978	0.0829
0.0003	71.0	1363697	0.9685	0.2942	0.0795

Framework versions

• Transformers 4.47.0
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.21.0