metadata

base_model: distilbert-base-uncased
library_name: peft
license: apache-2.0
metrics:
  - precision
  - recall
  - f1
  - accuracy
tags:
  - generated_from_trainer
model-index:
  - name: distilbert-ner-qlorafinetune-runs
    results: []

distilbert-ner-qlorafinetune-runs

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.1707
Precision: 0.9584
Recall: 0.9495
F1: 0.9539
Accuracy: 0.9737

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0004
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 640
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
4.6043	0.0151	20	1.9157	0.0	0.0	0.0	0.6192
1.5758	0.0303	40	0.7695	0.7538	0.4952	0.5977	0.8411
0.391	0.0454	60	0.3487	0.8781	0.8772	0.8777	0.9548
0.2109	0.0605	80	0.2782	0.8970	0.9301	0.9133	0.9655
0.1793	0.0756	100	0.2435	0.9635	0.9318	0.9474	0.9664
0.1055	0.0908	120	0.2311	0.9614	0.9330	0.9470	0.9667
0.3157	0.1059	140	0.2210	0.9631	0.9333	0.9480	0.9677
0.1085	0.1210	160	0.2088	0.9336	0.9364	0.9350	0.9692
0.2085	0.1362	180	0.2044	0.9576	0.9351	0.9462	0.9695
0.3833	0.1513	200	0.1992	0.9478	0.9402	0.9440	0.9703
0.097	0.1664	220	0.1957	0.9482	0.9426	0.9454	0.9712
0.12	0.1815	240	0.1964	0.9541	0.9421	0.9481	0.9716
0.1696	0.1967	260	0.2697	0.9315	0.9444	0.9379	0.9718
0.1405	0.2118	280	0.1933	0.9691	0.9424	0.9555	0.9717
0.1992	0.2269	300	0.1887	0.9538	0.9444	0.9491	0.9722
0.0907	0.2421	320	0.1870	0.9629	0.9441	0.9534	0.9724
0.1778	0.2572	340	0.1852	0.9461	0.9471	0.9466	0.9730
0.1474	0.2723	360	0.1821	0.9467	0.9472	0.9469	0.9731
0.1972	0.2874	380	0.1798	0.9522	0.9472	0.9497	0.9731
0.1807	0.3026	400	0.1823	0.9646	0.9465	0.9555	0.9729
0.1388	0.3177	420	0.1771	0.9474	0.9491	0.9482	0.9733
0.1664	0.3328	440	0.1762	0.9655	0.9470	0.9562	0.9732
0.1559	0.3480	460	0.1747	0.9618	0.9482	0.9550	0.9733
0.233	0.3631	480	0.1750	0.9663	0.9484	0.9573	0.9733
0.1851	0.3782	500	0.1738	0.9616	0.9492	0.9554	0.9735
0.2512	0.3933	520	0.1725	0.9642	0.9491	0.9566	0.9736
0.0823	0.4085	540	0.1721	0.9624	0.9490	0.9556	0.9736
0.0865	0.4236	560	0.1717	0.9624	0.9491	0.9557	0.9737
0.0611	0.4387	580	0.1714	0.9607	0.9493	0.9550	0.9738
0.105	0.4539	600	0.1710	0.9612	0.9493	0.9552	0.9737
0.1192	0.4690	620	0.1709	0.9584	0.9495	0.9540	0.9737
0.1923	0.4841	640	0.1707	0.9584	0.9495	0.9539	0.9737

Framework versions

PEFT 0.12.0
Transformers 4.43.3
Pytorch 2.4.1+cu121
Datasets 2.20.0
Tokenizers 0.19.1