asr-africa/mms-1B_all_NCHLT_AFRIKAANS_50hr_v1

facebook mms-1b-all Afrikaans - Beijuka Bruno

This model is a fine-tuned version of facebook/mms-1b-all on the NCHLT_speech_corpus/Afrikaans dataset. It achieves the following results on the evaluation set:

• Loss: 0.8746
• Wer: 0.7867
• Cer: 0.1929

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.5068	1.0	3678	0.0972	0.1899	0.0251
0.1819	2.0	7356	0.0805	0.1433	0.0203
0.1591	3.0	11034	0.0739	0.1270	0.0181
0.1449	4.0	14712	0.0700	0.1249	0.0174
0.1355	5.0	18390	0.0673	0.1135	0.0162
0.1283	6.0	22068	0.0655	0.1094	0.0153
0.1218	7.0	25746	0.0605	0.0953	0.0142
0.1171	8.0	29424	0.0580	0.0942	0.0141
0.113	9.0	33102	0.0576	0.0916	0.0132
0.1071	10.0	36780	0.0605	0.1099	0.0143
0.106	11.0	40458	0.0558	0.0878	0.0129
0.1021	12.0	44136	0.0542	0.0810	0.0122
0.0992	13.0	47814	0.0529	0.0807	0.0120
0.0965	14.0	51492	0.0542	0.0884	0.0124
0.0946	15.0	55170	0.0511	0.0798	0.0118
0.0914	16.0	58848	0.0500	0.0756	0.0112
0.0898	17.0	62526	0.0499	0.0774	0.0114
0.0868	18.0	66204	0.0484	0.0706	0.0108
0.085	19.0	69882	0.0495	0.0752	0.0109
0.0843	20.0	73560	0.0492	0.0704	0.0105
0.0822	21.0	77238	0.0480	0.0671	0.0102
0.0807	22.0	80916	0.0473	0.0728	0.0104
0.0794	23.0	84594	0.0462	0.0694	0.0102
0.0781	24.0	88272	0.0460	0.0668	0.0101
0.0771	25.0	91950	0.0463	0.0677	0.0100
0.0754	26.0	95628	0.0439	0.0598	0.0091
0.0745	27.0	99306	0.0463	0.0632	0.0097
0.0738	28.0	102984	0.0455	0.0635	0.0098
0.0718	29.0	106662	0.0455	0.0631	0.0095
0.0707	30.0	110340	0.0433	0.0591	0.0089
0.0696	31.0	114018	0.0454	0.0598	0.0095
0.0685	32.0	117696	0.0446	0.0613	0.0093
0.0674	33.0	121374	0.0437	0.0565	0.0088
0.0682	34.0	125052	0.0467	0.0679	0.0097
0.0648	35.0	128730	0.0467	0.0587	0.0090
0.0658	36.0	132408	0.0439	0.0535	0.0085
0.0633	37.0	136086	0.0448	0.0539	0.0085
0.0639	38.0	139764	0.0440	0.0569	0.0089
0.0619	39.0	143442	0.0458	0.0584	0.0088
0.0622	40.0	147120	0.0449	0.0587	0.0089
0.0618	41.0	150798	0.0461	0.0567	0.0087
0.0614	42.0	154476	0.0461	0.0550	0.0088
0.0611	43.0	158154	0.0427	0.0554	0.0085
0.0588	44.0	161832	0.0436	0.0529	0.0084
0.058	45.0	165510	0.0446	0.0546	0.0085
0.059	46.0	169188	0.0432	0.0543	0.0083

Framework versions

• Transformers 4.47.0
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0