distilhubert-audio-course-finetuned-gtzan-v5

This model is a fine-tuned version of ntu-spml/distilhubert on the GTZAN dataset. It achieves the following results on the evaluation set:

• Loss: 0.9236
• Accuracy: 0.87

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.7
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.2989	0.99	56	2.2882	0.11
2.2716	2.0	113	2.2469	0.31
2.1919	2.99	169	2.1317	0.4
2.0117	4.0	226	1.9244	0.53
1.7966	4.99	282	1.7315	0.65
1.6379	6.0	339	1.5920	0.59
1.4496	6.99	395	1.3539	0.71
1.3264	8.0	452	1.1879	0.7
1.0601	8.99	508	1.1342	0.7
0.9737	10.0	565	0.9209	0.79
0.7915	10.99	621	0.8768	0.74
0.6432	12.0	678	0.8060	0.8
0.5217	12.99	734	0.6562	0.85
0.3335	14.0	791	0.7744	0.76
0.2866	14.99	847	0.6969	0.82
0.1425	16.0	904	0.6378	0.82
0.1278	16.99	960	0.6972	0.82
0.0706	18.0	1017	0.7328	0.84
0.0301	18.99	1073	0.9245	0.76
0.0379	20.0	1130	0.8437	0.85
0.0147	20.99	1186	0.7489	0.83
0.0067	22.0	1243	0.8975	0.83
0.0049	22.99	1299	1.1788	0.81
0.0038	24.0	1356	1.1146	0.81
0.0028	24.99	1412	1.0270	0.85
0.0027	26.0	1469	1.0634	0.83
0.0024	26.99	1525	1.0220	0.84
0.0023	28.0	1582	1.0282	0.83
0.0487	28.99	1638	1.0735	0.82
0.0458	30.0	1695	1.1198	0.82
0.2453	30.99	1751	1.1154	0.81
0.0552	32.0	1808	1.1630	0.79
0.1202	32.99	1864	1.2746	0.81
0.2709	34.0	1921	1.3797	0.79
0.275	34.99	1977	1.5372	0.75
0.1268	36.0	2034	0.8140	0.86
0.1582	36.99	2090	1.4153	0.77
0.0054	38.0	2147	1.3796	0.79
0.0299	38.99	2203	1.3653	0.78
0.0199	40.0	2260	0.9987	0.87
0.0021	40.99	2316	1.0689	0.84
0.0007	42.0	2373	1.0383	0.85
0.0006	42.99	2429	1.0493	0.84
0.0006	44.0	2486	1.0744	0.85
0.0005	44.99	2542	0.9151	0.86
0.0004	46.0	2599	0.8946	0.87
0.01	46.99	2655	0.8960	0.88
0.0073	48.0	2712	0.9485	0.87
0.0004	48.99	2768	0.9247	0.87
0.0004	49.56	2800	0.9236	0.87

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1+cu117
• Datasets 2.14.4
• Tokenizers 0.13.3