DEREXP

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.1590
• Mse: 3.1590
• Mae: 1.3397
• R2: 0.4465
• Accuracy: 0.2528

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Mse	Mae	R2	Accuracy
14.7557	0.01	500	4.3307	4.3307	1.6240	0.2411	0.1976
4.5754	0.02	1000	4.1273	4.1273	1.5719	0.2768	0.2084
4.2925	0.02	1500	4.3074	4.3074	1.6155	0.2452	0.2012
3.9816	0.03	2000	3.7767	3.7767	1.5008	0.3382	0.2134
3.9171	0.04	2500	3.7033	3.7033	1.4732	0.3511	0.2304
3.946	0.05	3000	3.6217	3.6217	1.4552	0.3654	0.2352
4.1	0.06	3500	3.6101	3.6101	1.4612	0.3674	0.2216
3.8535	0.06	4000	3.6160	3.6160	1.4576	0.3664	0.2294
3.9037	0.07	4500	3.5864	3.5864	1.4476	0.3716	0.2374
3.9358	0.08	5000	3.5087	3.5087	1.4237	0.3852	0.2414
3.8062	0.09	5500	3.6085	3.6085	1.4595	0.3677	0.2256
3.8802	0.1	6000	3.6371	3.6371	1.4615	0.3627	0.223
3.7239	0.1	6500	3.5191	3.5191	1.4278	0.3834	0.2324
3.7618	0.11	7000	3.8408	3.8408	1.4973	0.3270	0.2316
3.7217	0.12	7500	3.8241	3.8241	1.5046	0.3299	0.2236
3.8204	0.13	8000	3.5290	3.5290	1.4256	0.3816	0.2388
3.7211	0.14	8500	3.6903	3.6903	1.4674	0.3534	0.227
3.7243	0.14	9000	3.4718	3.4718	1.4201	0.3917	0.231
3.7713	0.15	9500	3.8970	3.8970	1.5304	0.3171	0.2092
3.6289	0.16	10000	3.5273	3.5273	1.4255	0.3819	0.2388
3.7516	0.17	10500	3.9020	3.9020	1.5230	0.3163	0.2138
3.7491	0.18	11000	3.4809	3.4809	1.4209	0.3901	0.2378
3.7809	0.18	11500	3.8779	3.8779	1.5087	0.3205	0.229
3.7163	0.19	12000	3.5177	3.5177	1.4330	0.3836	0.2298
3.732	0.2	12500	3.9986	3.9986	1.5401	0.2993	0.218
3.7381	0.21	13000	3.4782	3.4782	1.4277	0.3905	0.2302
3.7652	0.22	13500	3.6239	3.6239	1.4587	0.3650	0.2244
3.6003	0.22	14000	3.4873	3.4873	1.4288	0.3889	0.2316
3.6865	0.23	14500	3.5895	3.5895	1.4511	0.3710	0.23
3.7398	0.24	15000	3.8835	3.8835	1.5183	0.3195	0.2172
3.5939	0.25	15500	3.6334	3.6334	1.4643	0.3633	0.2256
3.691	0.26	16000	3.4251	3.4251	1.3994	0.3998	0.2488
3.7279	0.26	16500	3.3956	3.3956	1.4034	0.4050	0.2336
3.797	0.27	17000	3.4029	3.4029	1.3968	0.4037	0.2486
3.684	0.28	17500	3.5831	3.5831	1.4451	0.3721	0.2304
3.5894	0.29	18000	3.6120	3.6120	1.4492	0.3671	0.2338
3.5938	0.3	18500	3.4975	3.4975	1.4240	0.3871	0.231
3.4948	0.3	19000	3.4791	3.4791	1.4167	0.3904	0.24
3.6527	0.31	19500	3.3409	3.3409	1.3817	0.4146	0.2474
3.5545	0.32	20000	3.3412	3.3412	1.3860	0.4145	0.2466
3.6102	0.33	20500	3.4148	3.4148	1.3961	0.4016	0.2488
3.542	0.34	21000	3.5980	3.5980	1.4508	0.3695	0.2244
3.5081	0.34	21500	3.6310	3.6310	1.4488	0.3637	0.2372
3.7745	0.35	22000	3.5246	3.5246	1.4294	0.3824	0.2378
3.5048	0.36	22500	3.4395	3.4395	1.4126	0.3973	0.241
3.6374	0.37	23000	3.3863	3.3863	1.3928	0.4066	0.247
3.5231	0.38	23500	3.5991	3.5991	1.4468	0.3693	0.2348
3.5893	0.38	24000	3.2910	3.2910	1.3692	0.4233	0.2504
3.5051	0.39	24500	3.3765	3.3765	1.3953	0.4083	0.2394
3.6082	0.4	25000	3.3060	3.3060	1.3830	0.4207	0.2412
3.4009	0.41	25500	3.4448	3.4448	1.4095	0.3964	0.2404
3.4239	0.42	26000	3.4127	3.4127	1.4027	0.4020	0.2412
3.6036	0.42	26500	3.5339	3.5339	1.4405	0.3808	0.2266
3.4107	0.43	27000	3.3319	3.3319	1.3776	0.4162	0.2542
3.3903	0.44	27500	3.4434	3.4434	1.4072	0.3966	0.2486
3.5583	0.45	28000	3.3119	3.3119	1.3728	0.4197	0.2516
3.4701	0.46	28500	3.3733	3.3733	1.3910	0.4089	0.2494
3.4113	0.46	29000	3.4144	3.4144	1.4027	0.4017	0.2414
3.5731	0.47	29500	3.3822	3.3822	1.3911	0.4073	0.2428
3.5738	0.48	30000	3.4408	3.4408	1.4120	0.3971	0.2386
3.481	0.49	30500	3.3255	3.3255	1.3794	0.4173	0.2514
3.4716	0.5	31000	3.2817	3.2817	1.3703	0.4250	0.2492
3.5487	0.5	31500	3.3388	3.3388	1.3851	0.4149	0.2472
3.2559	0.51	32000	3.3552	3.3552	1.3847	0.4121	0.249
3.5715	0.52	32500	3.2896	3.2896	1.3692	0.4236	0.251
3.4085	0.53	33000	3.2690	3.2690	1.3685	0.4272	0.2522
3.5582	0.54	33500	3.3228	3.3228	1.3800	0.4178	0.2462
3.4105	0.54	34000	3.4462	3.4462	1.4089	0.3961	0.2474
3.5401	0.55	34500	3.3181	3.3181	1.3751	0.4186	0.2558
3.4213	0.56	35000	3.2455	3.2455	1.3592	0.4313	0.2548
3.4644	0.57	35500	3.3900	3.3900	1.4004	0.4060	0.2388
3.4277	0.58	36000	3.2150	3.2150	1.3506	0.4366	0.2558
3.3376	0.58	36500	3.3522	3.3522	1.3944	0.4126	0.24
3.4311	0.59	37000	3.4152	3.4152	1.3980	0.4016	0.2498
3.336	0.6	37500	3.2996	3.2996	1.3674	0.4218	0.2594
3.3557	0.61	38000	3.2040	3.2040	1.3499	0.4386	0.2486
3.3586	0.62	38500	3.2784	3.2784	1.3632	0.4255	0.2534
3.3187	0.62	39000	3.3466	3.3466	1.3832	0.4136	0.2468
3.3899	0.63	39500	3.3209	3.3209	1.3795	0.4181	0.25
3.4483	0.64	40000	3.4685	3.4685	1.4165	0.3922	0.2436
3.3463	0.65	40500	3.3874	3.3874	1.3961	0.4064	0.2448
3.373	0.66	41000	3.2243	3.2243	1.3518	0.4350	0.2562
3.4526	0.66	41500	3.2819	3.2819	1.3693	0.4249	0.253
3.3581	0.67	42000	3.3412	3.3412	1.3843	0.4145	0.2456
3.4551	0.68	42500	3.2484	3.2484	1.3594	0.4308	0.2574
3.4022	0.69	43000	3.2010	3.2010	1.3468	0.4391	0.2568
3.3281	0.7	43500	3.3184	3.3184	1.3764	0.4185	0.2476
3.4044	0.7	44000	3.2361	3.2361	1.3528	0.4329	0.2506
3.3427	0.71	44500	3.2269	3.2269	1.3557	0.4346	0.2492
3.4106	0.72	45000	3.2758	3.2758	1.3733	0.4260	0.2434
3.4406	0.73	45500	3.2235	3.2235	1.3548	0.4352	0.2526
3.491	0.74	46000	3.2842	3.2842	1.3688	0.4245	0.2496
3.4671	0.74	46500	3.1811	3.1811	1.3464	0.4426	0.249
3.5774	0.75	47000	3.2649	3.2649	1.3608	0.4279	0.251
3.4953	0.76	47500	3.2681	3.2681	1.3616	0.4273	0.2538
3.4212	0.77	48000	3.4407	3.4407	1.4088	0.3971	0.2424
3.3285	0.78	48500	3.3279	3.3279	1.3771	0.4169	0.2454
3.361	0.78	49000	3.3717	3.3717	1.3910	0.4092	0.243
3.5419	0.79	49500	3.2851	3.2851	1.3748	0.4244	0.2448
3.3979	0.8	50000	3.3991	3.3991	1.4039	0.4044	0.2378
3.3354	0.81	50500	3.2636	3.2636	1.3650	0.4281	0.2456
3.4488	0.82	51000	3.2604	3.2604	1.3695	0.4287	0.243
3.2583	0.82	51500	3.2759	3.2759	1.3759	0.4260	0.2442
3.3419	0.83	52000	3.2789	3.2789	1.3728	0.4254	0.2494
3.4243	0.84	52500	3.2993	3.2993	1.3772	0.4219	0.2486
3.3154	0.85	53000	3.2350	3.2350	1.3585	0.4331	0.2528
3.3462	0.86	53500	3.2361	3.2361	1.3594	0.4329	0.2516
3.4554	0.86	54000	3.2307	3.2307	1.3548	0.4339	0.2528
3.5053	0.87	54500	3.1970	3.1970	1.3494	0.4398	0.2526
3.2745	0.88	55000	3.2506	3.2506	1.3614	0.4304	0.2546
3.3788	0.89	55500	3.2090	3.2090	1.3540	0.4377	0.2516
3.3216	0.9	56000	3.3347	3.3347	1.3857	0.4157	0.2462
3.2991	0.9	56500	3.1590	3.1590	1.3397	0.4465	0.2528
3.175	0.91	57000	3.2950	3.2950	1.3734	0.4226	0.2534
3.4697	0.92	57500	3.2021	3.2021	1.3483	0.4389	0.255
3.2413	0.93	58000	3.2157	3.2157	1.3523	0.4365	0.2518
3.3949	0.94	58500	3.2709	3.2709	1.3678	0.4268	0.2494
3.3502	0.94	59000	3.2263	3.2263	1.3558	0.4347	0.253
3.3492	0.95	59500	3.2667	3.2667	1.3659	0.4276	0.2538
3.3568	0.96	60000	3.1717	3.1717	1.3410	0.4442	0.2542
3.3886	0.97	60500	3.1800	3.1800	1.3444	0.4428	0.2534
3.2994	0.98	61000	3.2166	3.2166	1.3539	0.4364	0.2498
3.3381	0.98	61500	3.1964	3.1964	1.3484	0.4399	0.2534
3.351	0.99	62000	3.1664	3.1664	1.3393	0.4452	0.2538
3.4063	1.0	62500	3.1764	3.1764	1.3421	0.4434	0.2542

Framework versions

• Transformers 4.20.1
• Pytorch 1.12.0+cu113
• Datasets 2.3.2
• Tokenizers 0.12.1