galsenai/wavlm-large-waxal-keyword-spotting

wavlm-large

This model is a fine-tuned version of microsoft/wavlm-large on the galsenai/waxal_dataset dataset. It achieves the following results on the evaluation set:

• Loss: 0.5936
• Accuracy: 0.8950
• Precision: 0.9789
• F1: 0.9334

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 12
• eval_batch_size: 12
• seed: 0
• gradient_accumulation_steps: 4
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 32.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	F1
4.7405	1.01	500	5.1525	0.0	0.0	0.0
4.4299	2.02	1000	5.8969	0.0	0.0	0.0
4.2868	3.04	1500	4.9304	0.0019	0.0031	0.0023
3.6242	4.05	2000	4.3396	0.0409	0.0224	0.0237
2.686	5.06	2500	3.9399	0.0549	0.0320	0.0308
1.9284	6.07	3000	3.7736	0.0500	0.0779	0.0442
1.3936	7.08	3500	3.5380	0.0947	0.1381	0.0916
1.0764	8.1	4000	3.3281	0.1584	0.3514	0.1839
0.872	9.11	4500	2.9592	0.2755	0.6027	0.3315
0.7026	10.12	5000	2.5049	0.3971	0.6971	0.4587
0.603	11.13	5500	2.1485	0.5479	0.8074	0.6129
0.5042	12.15	6000	1.6532	0.7014	0.8604	0.7544
0.4542	13.16	6500	1.4057	0.7435	0.8941	0.7990
0.388	14.17	7000	1.2338	0.7802	0.9219	0.8332
0.3515	15.18	7500	0.9898	0.8170	0.9433	0.8681
0.3195	16.19	8000	1.1404	0.8067	0.9523	0.8635
0.2882	17.21	8500	0.9811	0.8177	0.9540	0.8746
0.2695	18.22	9000	0.9483	0.8318	0.9616	0.8878
0.2535	19.23	9500	0.6694	0.8844	0.9692	0.9198
0.2437	20.24	10000	0.7546	0.8700	0.9656	0.9125
0.2376	21.25	10500	0.6698	0.8810	0.9695	0.9202
0.2214	22.27	11000	0.7156	0.8727	0.9726	0.9174
0.2148	23.28	11500	0.5982	0.8931	0.9711	0.9286
0.2087	24.29	12000	0.7109	0.8814	0.9757	0.9243
0.2039	25.3	12500	0.6577	0.8897	0.9799	0.9306
0.1997	26.32	13000	0.7307	0.8746	0.9774	0.9203
0.1896	27.33	13500	0.6143	0.8905	0.9748	0.9290
0.1869	28.34	14000	0.6380	0.8909	0.9739	0.9287
0.185	29.35	14500	0.6932	0.8871	0.9791	0.9289
0.1813	30.36	15000	0.5936	0.8950	0.9789	0.9334
0.1801	31.38	15500	0.6150	0.8947	0.9801	0.9334

Framework versions

• Transformers 4.27.0.dev0
• Pytorch 1.11.0+cu113
• Datasets 2.9.1.dev0
• Tokenizers 0.13.2