silviacamplani/distilbert-finetuned-dapt_tapt-ner-music

This model is a fine-tuned version of silviacamplani/distilbert-finetuned-dapt_tapt-lm-ai on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.6073
• Validation Loss: 0.7078
• Train Precision: 0.5337
• Train Recall: 0.5986
• Train F1: 0.5643
• Train Accuracy: 0.8344
• Epoch: 9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'inner_optimizer': {'class_name': 'AdamWeightDecay', 'config': {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 1e-05, 'decay_steps': 370, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}}, 'dynamic': True, 'initial_scale': 32768.0, 'dynamic_growth_steps': 2000}
• training_precision: mixed_float16

Training results

Train Loss	Validation Loss	Train Precision	Train Recall	Train F1	Train Accuracy	Epoch
2.6231	2.0072	0.0	0.0	0.0	0.5482	0
1.7195	1.5337	0.1905	0.0072	0.0139	0.5597	1
1.3447	1.2423	0.3073	0.3510	0.3277	0.6910	2
1.1065	1.0569	0.4162	0.4536	0.4341	0.7195	3
0.9326	0.9225	0.5050	0.5473	0.5253	0.7689	4
0.8061	0.8345	0.5306	0.5770	0.5528	0.8011	5
0.7118	0.7749	0.5292	0.5878	0.5569	0.8176	6
0.6636	0.7366	0.5314	0.5950	0.5614	0.8242	7
0.6284	0.7158	0.5330	0.5968	0.5631	0.8321	8
0.6073	0.7078	0.5337	0.5986	0.5643	0.8344	9

Framework versions

• Transformers 4.20.1
• TensorFlow 2.6.4
• Datasets 2.1.0
• Tokenizers 0.12.1