arabert_cross_relevance_task6_fold6

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4430
• Qwk: 0.1465
• Mse: 0.4429

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1176	2	0.7830	0.0204	0.7810
No log	0.2353	4	0.3944	0.0715	0.3940
No log	0.3529	6	0.3437	0.0480	0.3436
No log	0.4706	8	0.3309	0.1105	0.3305
No log	0.5882	10	0.2835	0.1105	0.2836
No log	0.7059	12	0.2697	0.1105	0.2702
No log	0.8235	14	0.2695	0.1185	0.2700
No log	0.9412	16	0.2715	0.1326	0.2721
No log	1.0588	18	0.2745	0.2102	0.2750
No log	1.1765	20	0.2896	0.2067	0.2899
No log	1.2941	22	0.2766	0.2206	0.2770
No log	1.4118	24	0.2712	0.2420	0.2717
No log	1.5294	26	0.2809	0.2235	0.2814
No log	1.6471	28	0.2904	0.2076	0.2908
No log	1.7647	30	0.2687	0.2352	0.2692
No log	1.8824	32	0.2581	0.2477	0.2588
No log	2.0	34	0.2565	0.2472	0.2572
No log	2.1176	36	0.2641	0.2099	0.2646
No log	2.2353	38	0.3196	0.2033	0.3198
No log	2.3529	40	0.3095	0.2076	0.3097
No log	2.4706	42	0.2662	0.2084	0.2667
No log	2.5882	44	0.2644	0.2790	0.2651
No log	2.7059	46	0.2676	0.2311	0.2682
No log	2.8235	48	0.2943	0.2347	0.2946
No log	2.9412	50	0.3231	0.2150	0.3232
No log	3.0588	52	0.3309	0.2064	0.3309
No log	3.1765	54	0.3053	0.2252	0.3056
No log	3.2941	56	0.2808	0.2266	0.2812
No log	3.4118	58	0.2831	0.2170	0.2836
No log	3.5294	60	0.2875	0.2211	0.2881
No log	3.6471	62	0.3017	0.2140	0.3022
No log	3.7647	64	0.3078	0.2125	0.3084
No log	3.8824	66	0.3215	0.2117	0.3220
No log	4.0	68	0.3285	0.2052	0.3288
No log	4.1176	70	0.3679	0.1824	0.3679
No log	4.2353	72	0.3668	0.1824	0.3668
No log	4.3529	74	0.3216	0.2135	0.3219
No log	4.4706	76	0.2935	0.2301	0.2940
No log	4.5882	78	0.2944	0.2406	0.2949
No log	4.7059	80	0.3279	0.2200	0.3282
No log	4.8235	82	0.3629	0.1977	0.3631
No log	4.9412	84	0.3823	0.1898	0.3823
No log	5.0588	86	0.3659	0.1977	0.3659
No log	5.1765	88	0.3351	0.2019	0.3353
No log	5.2941	90	0.3442	0.2019	0.3443
No log	5.4118	92	0.3693	0.1977	0.3694
No log	5.5294	94	0.3861	0.1937	0.3861
No log	5.6471	96	0.3622	0.1977	0.3624
No log	5.7647	98	0.3369	0.2023	0.3373
No log	5.8824	100	0.3520	0.1939	0.3524
No log	6.0	102	0.3764	0.1898	0.3766
No log	6.1176	104	0.4008	0.1975	0.4009
No log	6.2353	106	0.4228	0.1899	0.4229
No log	6.3529	108	0.4377	0.1725	0.4376
No log	6.4706	110	0.4032	0.1824	0.4033
No log	6.5882	112	0.3828	0.1937	0.3829
No log	6.7059	114	0.4023	0.1975	0.4023
No log	6.8235	116	0.4098	0.1650	0.4097
No log	6.9412	118	0.4555	0.1465	0.4553
No log	7.0588	120	0.5148	0.1339	0.5144
No log	7.1765	122	0.5125	0.1339	0.5122
No log	7.2941	124	0.4645	0.1465	0.4643
No log	7.4118	126	0.3966	0.1751	0.3967
No log	7.5294	128	0.3450	0.1979	0.3452
No log	7.6471	130	0.3262	0.2210	0.3265
No log	7.7647	132	0.3261	0.2210	0.3265
No log	7.8824	134	0.3438	0.1979	0.3441
No log	8.0	136	0.3772	0.2015	0.3774
No log	8.1176	138	0.4155	0.1719	0.4155
No log	8.2353	140	0.4512	0.1528	0.4511
No log	8.3529	142	0.4592	0.1404	0.4590
No log	8.4706	144	0.4422	0.1465	0.4421
No log	8.5882	146	0.4170	0.1620	0.4170
No log	8.7059	148	0.4105	0.1757	0.4105
No log	8.8235	150	0.4205	0.1592	0.4204
No log	8.9412	152	0.4328	0.1465	0.4327
No log	9.0588	154	0.4368	0.1465	0.4367
No log	9.1765	156	0.4351	0.1465	0.4350
No log	9.2941	158	0.4352	0.1465	0.4352
No log	9.4118	160	0.4333	0.1465	0.4332
No log	9.5294	162	0.4373	0.1465	0.4372
No log	9.6471	164	0.4392	0.1465	0.4391
No log	9.7647	166	0.4394	0.1465	0.4393
No log	9.8824	168	0.4416	0.1465	0.4415
No log	10.0	170	0.4430	0.1465	0.4429

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1