distilbert-base-cased-airlines-news-multi-label-8-actions

This model is a fine-tuned version of distilbert/distilbert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2141
• F1: 0.9107
• Jaccard: 0.5749

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 150
• num_epochs: 45

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Jaccard
No log	1.0	76	0.4054	0.8583	0.4115
No log	2.0	152	0.2956	0.8583	0.4115
No log	3.0	228	0.2885	0.8583	0.4115
No log	4.0	304	0.2807	0.8583	0.4115
No log	5.0	380	0.2735	0.8583	0.4115
No log	6.0	456	0.2668	0.8583	0.4115
0.3504	7.0	532	0.2608	0.8709	0.4159
0.3504	8.0	608	0.2550	0.8822	0.4226
0.3504	9.0	684	0.2501	0.8930	0.4403
0.3504	10.0	760	0.2462	0.8941	0.4779
0.3504	11.0	836	0.2424	0.8907	0.4712
0.3504	12.0	912	0.2385	0.8899	0.4757
0.3504	13.0	988	0.2360	0.8930	0.5144
0.2673	14.0	1064	0.2335	0.8931	0.5081
0.2673	15.0	1140	0.2321	0.8921	0.5081
0.2673	16.0	1216	0.2292	0.8964	0.5380
0.2673	17.0	1292	0.2269	0.8975	0.5291
0.2673	18.0	1368	0.2256	0.8962	0.5461
0.2673	19.0	1444	0.2240	0.8968	0.5439
0.2471	20.0	1520	0.2237	0.8973	0.5454
0.2471	21.0	1596	0.2224	0.8971	0.5350
0.2471	22.0	1672	0.2213	0.8985	0.5313
0.2471	23.0	1748	0.2191	0.8982	0.5549
0.2471	24.0	1824	0.2186	0.9072	0.5542
0.2471	25.0	1900	0.2182	0.8993	0.5542
0.2471	26.0	1976	0.2168	0.9071	0.5608
0.2374	27.0	2052	0.2162	0.9091	0.5608
0.2374	28.0	2128	0.2154	0.8998	0.5601
0.2374	29.0	2204	0.2146	0.9095	0.5608
0.2374	30.0	2280	0.2141	0.9083	0.5608
0.2374	31.0	2356	0.2141	0.9107	0.5749
0.2374	32.0	2432	0.2136	0.9103	0.5704
0.2325	33.0	2508	0.2140	0.9052	0.5542
0.2325	34.0	2584	0.2125	0.9084	0.5608
0.2325	35.0	2660	0.2118	0.9091	0.5623
0.2325	36.0	2736	0.2118	0.9092	0.5667
0.2325	37.0	2812	0.2115	0.9072	0.5608
0.2325	38.0	2888	0.2118	0.9081	0.5667
0.2325	39.0	2964	0.2110	0.9080	0.5623
0.2293	40.0	3040	0.2111	0.9092	0.5667
0.2293	41.0	3116	0.2110	0.9087	0.5667
0.2293	42.0	3192	0.2109	0.9085	0.5623
0.2293	43.0	3268	0.2108	0.9092	0.5667
0.2293	44.0	3344	0.2107	0.9085	0.5623
0.2293	45.0	3420	0.2107	0.9085	0.5623

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1