distilbert-base-uncased_fold_6_ternary

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6625
• F1: 0.7588

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	292	0.5117	0.7306
0.5701	2.0	584	0.5273	0.7296
0.5701	3.0	876	0.6037	0.7415
0.2468	4.0	1168	0.7132	0.7318
0.2468	5.0	1460	0.8980	0.7504
0.12	6.0	1752	1.0343	0.7369
0.0486	7.0	2044	1.1860	0.7333
0.0486	8.0	2336	1.3348	0.7437
0.019	9.0	2628	1.3040	0.7561
0.019	10.0	2920	1.4649	0.7293
0.0152	11.0	3212	1.4870	0.7431
0.0078	12.0	3504	1.5668	0.7455
0.0078	13.0	3796	1.5280	0.7378
0.0091	14.0	4088	1.5672	0.7410
0.0091	15.0	4380	1.5948	0.7491
0.0052	16.0	4672	1.6625	0.7588
0.0052	17.0	4964	1.6544	0.7411
0.0048	18.0	5256	1.7124	0.7425
0.0024	19.0	5548	1.7211	0.7477
0.0024	20.0	5840	1.8216	0.7373
0.001	21.0	6132	1.8325	0.7361
0.001	22.0	6424	1.8089	0.7498
0.0015	23.0	6716	1.8026	0.7506
0.0005	24.0	7008	1.8026	0.7464
0.0005	25.0	7300	1.8043	0.7464

Framework versions

• Transformers 4.21.0
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1