mms-1b-toigen-female-model

This model is a fine-tuned version of facebook/mms-1b-all on the TOIGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.1967
• Wer: 0.3516

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.732	0.4032	100	3.5121	1.0008
2.379	0.8065	200	0.4962	0.6035
0.5414	1.2097	300	0.3402	0.5017
0.4817	1.6129	400	0.3003	0.4613
0.4129	2.0161	500	0.2658	0.4439
0.3635	2.4194	600	0.2473	0.4339
0.3872	2.8226	700	0.2446	0.4397
0.3373	3.2258	800	0.2411	0.4223
0.3222	3.6290	900	0.2304	0.4239
0.3738	4.0323	1000	0.2326	0.4177
0.3489	4.4355	1100	0.2286	0.4040
0.2941	4.8387	1200	0.2270	0.4181
0.3249	5.2419	1300	0.2203	0.4094
0.3202	5.6452	1400	0.2199	0.4065
0.312	6.0484	1500	0.2181	0.4023
0.2903	6.4516	1600	0.2151	0.3982
0.2897	6.8548	1700	0.2136	0.4019
0.2988	7.2581	1800	0.2123	0.3865
0.3326	7.6613	1900	0.2096	0.3853
0.2277	8.0645	2000	0.2191	0.3745
0.2715	8.4677	2100	0.2134	0.3928
0.3024	8.8710	2200	0.2068	0.3849
0.2667	9.2742	2300	0.2053	0.3782
0.2896	9.6774	2400	0.2046	0.3861
0.2296	10.0806	2500	0.2048	0.3774
0.2425	10.4839	2600	0.2003	0.3716
0.3036	10.8871	2700	0.1985	0.3662
0.2473	11.2903	2800	0.2006	0.3732
0.2589	11.6935	2900	0.1982	0.3620
0.2544	12.0968	3000	0.1982	0.3766
0.2551	12.5	3100	0.2029	0.3699
0.2136	12.9032	3200	0.1997	0.3645
0.2519	13.3065	3300	0.1960	0.3504
0.2769	13.7097	3400	0.1983	0.3508
0.2068	14.1129	3500	0.1985	0.3491
0.235	14.5161	3600	0.1967	0.3516

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0