mms-1b-bemgen-male-model

This model is a fine-tuned version of facebook/mms-1b-all on the GENBED - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.3017
• Wer: 0.4376

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
7.0099	0.1034	100	1.3602	0.9979
0.7457	0.2068	200	0.4181	0.5700
0.5383	0.3102	300	0.3908	0.5504
0.5398	0.4137	400	0.3656	0.5377
0.533	0.5171	500	0.3581	0.5182
0.4797	0.6205	600	0.3511	0.5016
0.5009	0.7239	700	0.3487	0.5051
0.4828	0.8273	800	0.3457	0.4883
0.4714	0.9307	900	0.3511	0.5154
0.4468	1.0341	1000	0.3369	0.4977
0.4507	1.1375	1100	0.3313	0.4593
0.4509	1.2410	1200	0.3317	0.4704
0.4772	1.3444	1300	0.3295	0.4672
0.4477	1.4478	1400	0.3239	0.4550
0.4947	1.5512	1500	0.3230	0.4683
0.4251	1.6546	1600	0.3249	0.4788
0.4554	1.7580	1700	0.3192	0.4704
0.424	1.8614	1800	0.3211	0.4485
0.436	1.9648	1900	0.3127	0.4358
0.4506	2.0683	2000	0.3163	0.4591
0.4595	2.1717	2100	0.3195	0.4404
0.4329	2.2751	2200	0.3125	0.4410
0.4438	2.3785	2300	0.3094	0.4419
0.4293	2.4819	2400	0.3088	0.4386
0.4097	2.5853	2500	0.3126	0.4374
0.4221	2.6887	2600	0.3093	0.4335
0.425	2.7921	2700	0.3063	0.4436
0.4027	2.8956	2800	0.3033	0.4187
0.4098	2.9990	2900	0.3060	0.4322
0.4025	3.1024	3000	0.3028	0.4204
0.4244	3.2058	3100	0.3035	0.4393
0.3814	3.3092	3200	0.3053	0.4227
0.4049	3.4126	3300	0.2998	0.4141
0.4055	3.5160	3400	0.3038	0.4210
0.4133	3.6194	3500	0.3045	0.4554
0.4083	3.7229	3600	0.3016	0.4376

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0