xlsr-he

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: 1.1042
• Wer: 0.5618
• Cer: 0.1990

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
4.8209	0.8368	100	4.9679	1.0	1.0
3.197	1.6736	200	3.5446	1.0	1.0
3.3296	2.5105	300	3.4076	1.0	1.0
2.3184	3.3473	400	2.7358	1.0137	0.6925
1.058	4.1841	500	1.4893	0.9039	0.4332
0.7087	5.0209	600	1.2311	0.8541	0.3847
0.5416	5.8577	700	1.0641	0.8256	0.3603
0.5273	6.6946	800	1.1168	0.8699	0.3384
0.5321	7.5314	900	1.1116	0.8200	0.3271
0.3195	8.3682	1000	1.0792	0.8480	0.3287
0.2597	9.2050	1100	1.0042	0.7433	0.3017
0.3101	10.0418	1200	1.0282	0.7255	0.2903
0.2546	10.8787	1300	0.9019	0.7265	0.2759
0.1811	11.7155	1400	1.0079	0.7107	0.2734
0.1583	12.5523	1500	1.1007	0.7356	0.2847
0.1904	13.3891	1600	1.1371	0.6843	0.2708
0.1575	14.2259	1700	1.1959	0.6889	0.2704
0.1265	15.0628	1800	1.1985	0.7428	0.2853
0.113	15.8996	1900	1.1250	0.6762	0.2611
0.17	16.7364	2000	1.1427	0.6980	0.2695
0.1124	17.5732	2100	1.1954	0.7062	0.2720
0.1232	18.4100	2200	1.1302	0.6706	0.2495
0.1115	19.2469	2300	1.1399	0.6741	0.2604
0.1016	20.0837	2400	1.1784	0.6441	0.2436
0.1455	20.9205	2500	1.2145	0.6751	0.2485
0.074	21.7573	2600	1.2742	0.6304	0.2545
0.0654	22.5941	2700	1.0912	0.6146	0.2266
0.1123	23.4310	2800	1.0622	0.6350	0.2352
0.1563	24.2678	2900	1.0540	0.6416	0.2318
0.0716	25.1046	3000	1.0769	0.6304	0.2331
0.1085	25.9414	3100	1.0909	0.5979	0.2212
0.1026	26.7782	3200	1.1154	0.6075	0.2227
0.0872	27.6151	3300	1.1303	0.6268	0.2301
0.0761	28.4519	3400	1.0749	0.6187	0.2300
0.1127	29.2887	3500	1.0718	0.6121	0.2205
0.0598	30.1255	3600	1.1657	0.6223	0.2264
0.096	30.9623	3700	1.1789	0.5938	0.2179
0.075	31.7992	3800	1.2492	0.6380	0.2350
0.057	32.6360	3900	1.2094	0.6263	0.2274
0.1077	33.4728	4000	1.1825	0.6040	0.2209
0.0455	34.3096	4100	1.1660	0.6136	0.2201
0.0688	35.1464	4200	1.1257	0.6050	0.2191
0.064	35.9833	4300	1.1346	0.5867	0.2122
0.0383	36.8201	4400	1.1975	0.6009	0.2180
0.0397	37.6569	4500	1.1697	0.5989	0.2177
0.0447	38.4937	4600	1.2174	0.6040	0.2186
0.0545	39.3305	4700	1.1958	0.5994	0.2184
0.0531	40.1674	4800	1.1707	0.5902	0.2127
0.1481	41.0042	4900	1.1475	0.5770	0.2071
0.0427	41.8410	5000	1.1719	0.5892	0.2114
0.0242	42.6778	5100	1.0977	0.5740	0.2053
0.0398	43.5146	5200	1.1135	0.5694	0.2063
0.037	44.3515	5300	1.1234	0.5735	0.2086
0.0255	45.1883	5400	1.1127	0.5740	0.2040
0.0467	46.0251	5500	1.1123	0.5653	0.1998
0.0261	46.8619	5600	1.1039	0.5719	0.2008
0.0512	47.6987	5700	1.1167	0.5679	0.2011
0.0551	48.5356	5800	1.1126	0.5648	0.1990
0.0373	49.3724	5900	1.1042	0.5618	0.1990

Framework versions

• Transformers 4.42.0.dev0
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1