wav2vec2-large-xls-r-300m-Arabic-phoneme-based-MDD-experiment3

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0558
• Per: 0.0156
• Wer: 0.0184

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 6
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 250
• num_epochs: 30.0

Training results

Training Loss	Epoch	Step	Validation Loss	Per	Wer
2.4598	1.0	546	1.8270	0.9903	0.9822
1.8808	2.0	1093	1.8033	0.9898	0.9813
1.8584	3.0	1640	1.7560	0.9950	0.9906
1.8313	4.0	2187	1.6726	0.9881	0.9817
1.764	5.0	2733	1.4356	0.9688	0.9680
1.5369	6.0	3280	0.6934	0.5617	0.5871
1.228	7.0	3827	0.3561	0.2158	0.2483
1.0419	8.0	4374	0.1982	0.0925	0.1170
0.9134	9.0	4920	0.1557	0.0692	0.0806
0.813	10.0	5467	0.1070	0.0507	0.0589
0.7471	11.0	6014	0.0969	0.0389	0.0453
0.693	12.0	6561	0.0840	0.0311	0.0375
0.6506	13.0	7107	0.0745	0.0285	0.0339
0.6159	14.0	7654	0.0765	0.0264	0.0321
0.5839	15.0	8201	0.0652	0.0253	0.0307
0.5578	16.0	8748	0.0672	0.0236	0.0281
0.5259	17.0	9294	0.0664	0.0231	0.0270
0.5011	18.0	9841	0.0656	0.0215	0.0254
0.4755	19.0	10388	0.0674	0.0206	0.0239
0.4471	20.0	10935	0.0649	0.0203	0.0242
0.4276	21.0	11481	0.0636	0.0202	0.0229
0.4065	22.0	12028	0.0589	0.0183	0.0214
0.3889	23.0	12575	0.0554	0.0179	0.0210
0.3695	24.0	13122	0.0572	0.0171	0.0202
0.3536	25.0	13668	0.0551	0.0167	0.0189
0.3385	26.0	14215	0.0604	0.0169	0.0197
0.3276	27.0	14762	0.0581	0.0157	0.0183
0.3174	28.0	15309	0.0564	0.0157	0.0181
0.3093	29.0	15855	0.0564	0.0153	0.0180
0.3017	29.96	16380	0.0558	0.0156	0.0184

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 1.18.3
• Tokenizers 0.13.3