gopdataset_base_fadam

This model is a fine-tuned version of facebook/wav2vec2-base-960h on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1001
• Wer: 0.1662

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.9033	1.05	500	3.0006	1.0
2.1655	2.11	1000	0.3024	0.3575
0.582	3.16	1500	0.2045	0.2566
0.3618	4.21	2000	0.1377	0.2188
0.3297	5.26	2500	0.1551	0.2232
0.2847	6.32	3000	0.1742	0.2486
0.2488	7.37	3500	0.2328	0.2036
0.1996	8.42	4000	0.1379	0.2079
0.2165	9.47	4500	0.1183	0.1924
0.189	10.53	5000	0.1295	0.1956
0.1817	11.58	5500	0.1198	0.1888
0.1682	12.63	6000	0.1270	0.1887
0.1246	13.68	6500	0.1211	0.1867
0.1442	14.74	7000	0.1301	0.1805
0.1732	15.79	7500	0.1107	0.1801
0.1142	16.84	8000	0.1096	0.1848
0.1485	17.89	8500	0.1075	0.1790
0.1037	18.95	9000	0.1109	0.1778
0.1155	20.0	9500	0.1120	0.1736
0.1162	21.05	10000	0.1053	0.1740
0.0874	22.11	10500	0.1157	0.1739
0.0797	23.16	11000	0.1128	0.1735
0.0726	24.21	11500	0.1089	0.1745
0.0691	25.26	12000	0.1084	0.1696
0.0677	26.32	12500	0.1059	0.1696
0.0582	27.37	13000	0.1065	0.1696
0.0669	28.42	13500	0.1001	0.1701
0.0956	29.47	14000	0.1017	0.1687

Framework versions

• Transformers 4.17.0
• Pytorch 2.5.1+cu121
• Datasets 1.18.3
• Tokenizers 0.20.3