scenario-kd-pre-ner-full-mdeberta-halfen_data-univner_en66

This model is a fine-tuned version of microsoft/mdeberta-v3-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 62.5095
• Precision: 0.7726
• Recall: 0.7598
• F1: 0.7662
• Accuracy: 0.9809

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 32
• seed: 66
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
134.1825	1.28	500	94.2693	0.5467	0.4058	0.4658	0.9615
82.4937	2.55	1000	77.0201	0.6945	0.6801	0.6872	0.9769
71.8548	3.83	1500	71.0742	0.7233	0.7277	0.7255	0.9791
66.6145	5.1	2000	67.2696	0.7376	0.7681	0.7525	0.9808
63.2728	6.38	2500	65.0104	0.7383	0.7329	0.7356	0.9796
61.2443	7.65	3000	63.3569	0.7767	0.7453	0.7607	0.9808
59.852	8.93	3500	62.5095	0.7726	0.7598	0.7662	0.9809

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3