asr-africa/mms-1B_all_NCHLT_XHOSA_50hr_v1

facebook mms-1b-all xhosa - Beijuka Bruno

This model is a fine-tuned version of facebook/mms-1b-all on the NCHLT_speech_corpus/Xhosa dataset. It achieves the following results on the evaluation set:

• Loss: 0.5077
• Model Preparation Time: 0.0185
• Wer: 0.7416
• Cer: 0.1466

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
3.0339	1.0	2604	0.2173	0.0185	0.3580	0.0532
1.074	2.0	5208	0.2030	0.0185	0.3267	0.0492
1.007	3.0	7812	0.2000	0.0185	0.3209	0.0485
0.9601	4.0	10416	0.1937	0.0185	0.3047	0.0468
0.9269	5.0	13020	0.1908	0.0185	0.3071	0.0460
0.8968	6.0	15624	0.1912	0.0185	0.3012	0.0455
0.8704	7.0	18228	0.1900	0.0185	0.2990	0.0450
0.854	8.0	20832	0.1900	0.0185	0.2993	0.0452
0.8287	9.0	23436	0.1888	0.0185	0.3047	0.0454
0.8122	10.0	26040	0.1881	0.0185	0.2976	0.0445
0.7944	11.0	28644	0.1850	0.0185	0.2916	0.0437
0.776	12.0	31248	0.1807	0.0185	0.2820	0.0426
0.7557	13.0	33852	0.1862	0.0185	0.2858	0.0429
0.7484	14.0	36456	0.1818	0.0185	0.2845	0.0428
0.7347	15.0	39060	0.1842	0.0185	0.2845	0.0434
0.7262	16.0	41664	0.1830	0.0185	0.2873	0.0434
0.713	17.0	44268	0.1829	0.0185	0.2850	0.0431
0.6957	18.0	46872	0.1829	0.0185	0.2803	0.0420
0.6867	19.0	49476	0.1813	0.0185	0.2750	0.0414
0.6716	20.0	52080	0.1812	0.0185	0.2743	0.0416
0.668	21.0	54684	0.1838	0.0185	0.2773	0.0423
0.662	22.0	57288	0.1875	0.0185	0.2743	0.0418
0.6471	23.0	59892	0.1828	0.0185	0.2779	0.0422
0.6414	24.0	62496	0.1863	0.0185	0.2770	0.0416
0.6322	25.0	65100	0.1851	0.0185	0.2754	0.0412
0.6238	26.0	67704	0.1847	0.0185	0.2811	0.0422
0.6151	27.0	70308	0.1830	0.0185	0.2682	0.0408
0.6046	28.0	72912	0.1811	0.0185	0.2700	0.0408
0.5973	29.0	75516	0.1841	0.0185	0.2694	0.0412
0.5897	30.0	78120	0.1826	0.0185	0.2653	0.0404
0.5855	31.0	80724	0.1826	0.0185	0.2653	0.0407
0.5738	32.0	83328	0.1801	0.0185	0.2712	0.0411
0.5664	33.0	85932	0.1837	0.0185	0.2682	0.0408
0.5607	34.0	88536	0.1870	0.0185	0.2646	0.0405
0.5556	35.0	91140	0.1848	0.0185	0.2665	0.0407
0.5493	36.0	93744	0.1822	0.0185	0.2597	0.0395
0.5426	37.0	96348	0.1833	0.0185	0.2693	0.0405
0.536	38.0	98952	0.1843	0.0185	0.2672	0.0404
0.5231	39.0	101556	0.1833	0.0185	0.2696	0.0405
0.5204	40.0	104160	0.1833	0.0185	0.2663	0.0401
0.5167	41.0	106764	0.1818	0.0185	0.2637	0.0395
0.513	42.0	109368	0.1842	0.0185	0.2643	0.0396
0.506	43.0	111972	0.1876	0.0185	0.2661	0.0401
0.4976	44.0	114576	0.1890	0.0185	0.2615	0.0400
0.4904	45.0	117180	0.1842	0.0185	0.2587	0.0392
0.4864	46.0	119784	0.1886	0.0185	0.2600	0.0391
0.4814	47.0	122388	0.1849	0.0185	0.2627	0.0398
0.4821	48.0	124992	0.1860	0.0185	0.2583	0.0390
0.4662	49.0	127596	0.1906	0.0185	0.2624	0.0397
0.4632	50.0	130200	0.1881	0.0185	0.2603	0.0393
0.465	51.0	132804	0.1876	0.0185	0.2627	0.0396
0.4549	52.0	135408	0.1898	0.0185	0.2596	0.0393
0.4532	53.0	138012	0.1916	0.0185	0.2606	0.0395
0.4478	54.0	140616	0.1912	0.0185	0.2617	0.0398
0.4418	55.0	143220	0.1935	0.0185	0.2600	0.0393

Framework versions

• Transformers 4.47.1
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0