distilhubert_multilabel_audioset_subset_50epochs

This model is a fine-tuned version of ntu-spml/distilhubert on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1095
• Accuracy: 0.9813
• F1: 0.1445
• Precision: 0.2747
• Recall: 0.0980

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
0.0904	1.0	257	0.0826	0.9839	0.0	0.0	0.0
0.0784	2.0	514	0.0808	0.9839	0.0	0.0	0.0
0.0816	3.0	771	0.0806	0.9839	0.0	0.0	0.0
0.0775	4.0	1028	0.0802	0.9839	0.0	0.0	0.0
0.0748	5.0	1285	0.0795	0.9839	0.0	0.0	0.0
0.0796	6.0	1542	0.0783	0.9839	0.0	0.0	0.0
0.0789	7.0	1799	0.0774	0.9839	0.0	0.0	0.0
0.0835	8.0	2056	0.0760	0.9839	0.0	0.0	0.0
0.0698	9.0	2313	0.0751	0.9839	0.0057	0.9048	0.0029
0.0715	10.0	2570	0.0738	0.9839	0.0170	0.6477	0.0086
0.0693	11.0	2827	0.0731	0.9839	0.0190	0.6809	0.0096
0.0665	12.0	3084	0.0732	0.9839	0.0202	0.6476	0.0103
0.0645	13.0	3341	0.0725	0.9838	0.0415	0.4675	0.0217
0.0633	14.0	3598	0.0724	0.9838	0.0479	0.4841	0.0252
0.0574	15.0	3855	0.0724	0.9836	0.0585	0.4066	0.0315
0.0608	16.0	4112	0.0731	0.9836	0.0614	0.4143	0.0332
0.0541	17.0	4369	0.0736	0.9836	0.0733	0.4263	0.0401
0.0554	18.0	4626	0.0738	0.9836	0.0712	0.4041	0.0390
0.0501	19.0	4883	0.0745	0.9836	0.0758	0.4233	0.0416
0.0533	20.0	5140	0.0759	0.9836	0.0788	0.4111	0.0436
0.0415	21.0	5397	0.0768	0.9836	0.0820	0.4131	0.0455
0.047	22.0	5654	0.0778	0.9835	0.0939	0.4079	0.0531
0.0451	23.0	5911	0.0797	0.9831	0.0979	0.3533	0.0568
0.0422	24.0	6168	0.0805	0.9829	0.1079	0.3408	0.0641
0.0419	25.0	6425	0.0823	0.9826	0.1106	0.3119	0.0672
0.0378	26.0	6682	0.0837	0.9826	0.1132	0.3167	0.0689
0.0439	27.0	6939	0.0858	0.9823	0.1231	0.3104	0.0767
0.0357	28.0	7196	0.0869	0.9828	0.1086	0.3352	0.0648
0.0375	29.0	7453	0.0882	0.9826	0.1157	0.3248	0.0704
0.0378	30.0	7710	0.0906	0.9826	0.1121	0.3159	0.0681
0.0355	31.0	7967	0.0923	0.9825	0.1179	0.3122	0.0727
0.0307	32.0	8224	0.0938	0.9825	0.1174	0.3137	0.0722
0.0293	33.0	8481	0.0948	0.9820	0.1298	0.2949	0.0832
0.0323	34.0	8738	0.0965	0.9819	0.1325	0.2942	0.0855
0.0271	35.0	8995	0.0972	0.9822	0.1314	0.3124	0.0832
0.0281	36.0	9252	0.0990	0.9818	0.1392	0.2949	0.0911
0.0254	37.0	9509	0.1003	0.9816	0.1362	0.2809	0.0899
0.0277	38.0	9766	0.1012	0.9820	0.1325	0.2995	0.0850
0.0263	39.0	10023	0.1025	0.9817	0.1437	0.2936	0.0951
0.0235	40.0	10280	0.1043	0.9820	0.1344	0.3005	0.0865
0.024	41.0	10537	0.1050	0.9816	0.1419	0.2860	0.0944
0.0256	42.0	10794	0.1057	0.9815	0.1382	0.2778	0.0920
0.0251	43.0	11051	0.1068	0.9816	0.1380	0.2846	0.0911
0.025	44.0	11308	0.1077	0.9815	0.1401	0.2806	0.0933
0.0235	45.0	11565	0.1083	0.9814	0.1434	0.2790	0.0965
0.0219	46.0	11822	0.1088	0.9813	0.1405	0.2712	0.0948
0.0222	47.0	12079	0.1088	0.9813	0.1443	0.2773	0.0975
0.023	48.0	12336	0.1093	0.9812	0.1452	0.2730	0.0989
0.0208	49.0	12593	0.1095	0.9813	0.1442	0.2749	0.0977
0.0211	50.0	12850	0.1095	0.9813	0.1445	0.2747	0.0980

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.0
• Datasets 2.19.1
• Tokenizers 0.19.1