hamsa-pretrained

This model is a fine-tuned version of nadsoft/Hamsa-large-v0.1-beta on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4344
• Wer: 29.2057

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00025
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 35000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.895	0.01	1000	1.8765	86.7012
1.6569	0.01	2000	1.5809	84.0907
1.3312	0.02	3000	1.3458	75.7090
1.2369	0.02	4000	1.2389	73.1365
1.1518	0.03	5000	1.1097	66.8170
1.0135	0.03	6000	1.0616	65.1843
1.0965	0.04	7000	1.0084	65.8582
0.867	0.04	8000	0.9305	57.6093
0.9425	0.05	9000	0.8907	55.4854
0.9501	0.05	10000	0.8393	54.0212
0.8602	0.06	11000	0.8096	53.4968
0.7596	0.06	12000	0.7761	51.9305
0.7334	0.07	13000	0.7694	49.4411
0.708	0.07	14000	0.7336	47.0040
0.7112	0.08	15000	0.7149	47.5783
0.6989	0.08	16000	0.6713	44.2986
0.7025	0.09	17000	0.6639	43.7481
0.6127	0.09	18000	0.6477	42.9127
0.6342	0.1	19000	0.6298	42.6826
0.6174	0.1	20000	0.6080	40.1172
0.5551	0.11	21000	0.5896	39.0398
0.5353	0.11	22000	0.5753	39.1253
0.5528	0.12	23000	0.5588	40.2881
0.5423	0.12	24000	0.5445	35.6606
0.5069	0.13	25000	0.5304	35.9358
0.4356	0.13	26000	0.5187	34.4930
0.5111	0.14	27000	0.5035	33.4227
0.5613	0.14	28000	0.4912	33.0952
0.4165	0.15	29000	0.4825	32.0155
0.4736	0.15	30000	0.4716	32.0914
0.4213	0.16	31000	0.4618	31.6026
0.4242	0.16	32000	0.4514	30.3757
0.3837	0.17	33000	0.4448	30.3116
0.4321	0.17	34000	0.4377	29.4691
0.4268	0.18	35000	0.4344	29.2057

Framework versions

• Transformers 4.37.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0