hamsa-small-finetuned-qasr

This model is a fine-tuned version of nadsoft/hamsa_small on the nadsoft/QASR-Speech-Resource default dataset. It achieves the following results on the evaluation set:

• Loss: 0.2831
• Wer: 22.5872

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 20000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.3207	0.03	2500	0.3458	24.4928
0.3264	0.05	5000	0.3316	23.4082
0.3223	0.08	7500	0.3239	25.1050
0.3121	0.1	10000	0.3143	23.4557
0.3103	0.13	12500	0.3079	23.4296
0.3041	0.15	15000	0.3033	23.2113
0.3077	0.18	17500	0.2998	21.7091
0.2867	0.2	20000	0.2943	20.1761
0.265	0.23	22500	0.2921	21.6522
0.3096	0.25	25000	0.2894	22.1505
0.2813	0.28	27500	0.2863	22.4993
0.2805	0.3	30000	0.2832	21.0114

Framework versions

• Transformers 4.37.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0