asr-africa/mms-1B_all_DigitalUmuganda_Afrivoice_Shona_86hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the DigitalUmuganda dataset. It achieves the following results on the evaluation set:

• Loss: 0.2009
• Model Preparation Time: 0.0179
• Wer: 0.2505
• Cer: 0.0456

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 150
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
3.4446	0.9997	940	0.2042	0.0179	0.2927	0.0496
0.881	1.9995	1880	0.1954	0.0179	0.2771	0.0467
0.833	2.9992	2820	0.1863	0.0179	0.2646	0.0444
0.8	4.0	3761	0.1833	0.0179	0.2636	0.0441
0.7753	4.9997	4701	0.1800	0.0179	0.2594	0.0430
0.7581	5.9995	5641	0.1776	0.0179	0.2558	0.0423
0.742	6.9992	6581	0.1746	0.0179	0.2549	0.0421
0.7279	8.0	7522	0.1733	0.0179	0.2515	0.0415
0.7167	8.9997	8462	0.1728	0.0179	0.2494	0.0408
0.7039	9.9995	9402	0.1706	0.0179	0.2461	0.0405
0.6943	10.9992	10342	0.1701	0.0179	0.2499	0.0403
0.6868	12.0	11283	0.1692	0.0179	0.2461	0.0403
0.6759	12.9997	12223	0.1686	0.0179	0.2424	0.0401
0.6664	13.9995	13163	0.1692	0.0179	0.2425	0.0399
0.6592	14.9992	14103	0.1664	0.0179	0.2442	0.0397
0.6538	16.0	15044	0.1661	0.0179	0.2404	0.0397
0.6467	16.9997	15984	0.1662	0.0179	0.2371	0.0390
0.6398	17.9995	16924	0.1670	0.0179	0.2383	0.0390
0.6369	18.9992	17864	0.1676	0.0179	0.2392	0.0393
0.6298	20.0	18805	0.1668	0.0179	0.2363	0.0386
0.6251	20.9997	19745	0.1676	0.0179	0.2406	0.0390
0.618	21.9995	20685	0.1656	0.0179	0.2383	0.0389
0.6091	22.9992	21625	0.1661	0.0179	0.2373	0.0391
0.607	24.0	22566	0.1655	0.0179	0.2405	0.0394
0.6036	24.9997	23506	0.1653	0.0179	0.2407	0.0392
0.5955	25.9995	24446	0.1653	0.0179	0.2402	0.0390
0.5915	26.9992	25386	0.1650	0.0179	0.2380	0.0389

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.0.2
• Tokenizers 0.20.1