distilbert-base-uncased_fold_10_ternary_v1

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.9887
• F1: 0.7797

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	290	0.5701	0.7463
0.5651	2.0	580	0.5359	0.7748
0.5651	3.0	870	0.6043	0.7847
0.2605	4.0	1160	1.0124	0.7587
0.2605	5.0	1450	1.1140	0.7599
0.1223	6.0	1740	1.2713	0.7859
0.0469	7.0	2030	1.3188	0.7822
0.0469	8.0	2320	1.3819	0.7946
0.0279	9.0	2610	1.5444	0.7847
0.0279	10.0	2900	1.5851	0.7908
0.0084	11.0	3190	1.7003	0.7822
0.0084	12.0	3480	1.8148	0.7748
0.007	13.0	3770	1.7651	0.7748
0.008	14.0	4060	1.8423	0.7748
0.008	15.0	4350	1.7871	0.7809
0.0054	16.0	4640	1.9324	0.7748
0.0054	17.0	4930	1.8685	0.7809
0.0048	18.0	5220	1.9901	0.7797
0.002	19.0	5510	1.9273	0.7785
0.002	20.0	5800	1.9945	0.7809
0.0018	21.0	6090	1.9250	0.7785
0.0018	22.0	6380	1.9929	0.7822
0.0032	23.0	6670	1.9306	0.7859
0.0032	24.0	6960	1.9603	0.7847
0.0029	25.0	7250	1.9887	0.7797

Framework versions

• Transformers 4.21.0
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1