arabert_cross_relevance_task4_fold3

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3037
• Qwk: 0.3732
• Mse: 0.3037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1176	2	0.5630	0.1562	0.5630
No log	0.2353	4	0.4644	0.2174	0.4644
No log	0.3529	6	0.3950	0.1962	0.3950
No log	0.4706	8	0.3834	0.2684	0.3834
No log	0.5882	10	0.3306	0.3081	0.3306
No log	0.7059	12	0.3268	0.2811	0.3268
No log	0.8235	14	0.3154	0.3011	0.3154
No log	0.9412	16	0.3060	0.3218	0.3060
No log	1.0588	18	0.3117	0.3651	0.3117
No log	1.1765	20	0.3057	0.3773	0.3057
No log	1.2941	22	0.3034	0.3911	0.3034
No log	1.4118	24	0.2791	0.4002	0.2791
No log	1.5294	26	0.2627	0.4023	0.2627
No log	1.6471	28	0.2813	0.3853	0.2813
No log	1.7647	30	0.3082	0.3740	0.3082
No log	1.8824	32	0.2928	0.3614	0.2928
No log	2.0	34	0.2792	0.3430	0.2792
No log	2.1176	36	0.2640	0.3836	0.2640
No log	2.2353	38	0.2797	0.4090	0.2797
No log	2.3529	40	0.3047	0.3991	0.3047
No log	2.4706	42	0.3233	0.3483	0.3233
No log	2.5882	44	0.3173	0.3975	0.3173
No log	2.7059	46	0.2794	0.3702	0.2794
No log	2.8235	48	0.2545	0.3595	0.2545
No log	2.9412	50	0.2598	0.3274	0.2598
No log	3.0588	52	0.3170	0.2780	0.3170
No log	3.1765	54	0.3616	0.2701	0.3616
No log	3.2941	56	0.3399	0.3799	0.3399
No log	3.4118	58	0.2912	0.4991	0.2912
No log	3.5294	60	0.2788	0.5808	0.2788
No log	3.6471	62	0.2781	0.5944	0.2781
No log	3.7647	64	0.2655	0.5547	0.2655
No log	3.8824	66	0.2739	0.4479	0.2739
No log	4.0	68	0.3052	0.3580	0.3052
No log	4.1176	70	0.3257	0.2703	0.3257
No log	4.2353	72	0.3077	0.2765	0.3077
No log	4.3529	74	0.2673	0.3446	0.2673
No log	4.4706	76	0.2558	0.4205	0.2558
No log	4.5882	78	0.2699	0.4751	0.2699
No log	4.7059	80	0.3058	0.5205	0.3058
No log	4.8235	82	0.3248	0.5030	0.3248
No log	4.9412	84	0.3083	0.4300	0.3083
No log	5.0588	86	0.2844	0.3512	0.2844
No log	5.1765	88	0.2595	0.3405	0.2595
No log	5.2941	90	0.2544	0.3462	0.2544
No log	5.4118	92	0.2686	0.3378	0.2686
No log	5.5294	94	0.3037	0.3446	0.3037
No log	5.6471	96	0.3453	0.3453	0.3453
No log	5.7647	98	0.3438	0.3956	0.3438
No log	5.8824	100	0.3084	0.4184	0.3084
No log	6.0	102	0.2757	0.4015	0.2757
No log	6.1176	104	0.2744	0.3963	0.2744
No log	6.2353	106	0.2900	0.4343	0.2900
No log	6.3529	108	0.3167	0.4357	0.3167
No log	6.4706	110	0.3249	0.4225	0.3249
No log	6.5882	112	0.3104	0.3929	0.3104
No log	6.7059	114	0.2965	0.3961	0.2965
No log	6.8235	116	0.2870	0.3956	0.2870
No log	6.9412	118	0.2835	0.4017	0.2835
No log	7.0588	120	0.2798	0.4117	0.2798
No log	7.1765	122	0.2952	0.3838	0.2952
No log	7.2941	124	0.3094	0.3838	0.3094
No log	7.4118	126	0.3254	0.4226	0.3254
No log	7.5294	128	0.3223	0.4122	0.3223
No log	7.6471	130	0.3091	0.3910	0.3091
No log	7.7647	132	0.3042	0.3896	0.3042
No log	7.8824	134	0.2977	0.3822	0.2977
No log	8.0	136	0.2987	0.3536	0.2987
No log	8.1176	138	0.3110	0.3437	0.3110
No log	8.2353	140	0.3121	0.3392	0.3121
No log	8.3529	142	0.3056	0.3402	0.3056
No log	8.4706	144	0.3047	0.3462	0.3047
No log	8.5882	146	0.3035	0.3572	0.3035
No log	8.7059	148	0.3024	0.3690	0.3024
No log	8.8235	150	0.3034	0.3717	0.3034
No log	8.9412	152	0.3012	0.3774	0.3012
No log	9.0588	154	0.2988	0.3774	0.2988
No log	9.1765	156	0.3009	0.3774	0.3009
No log	9.2941	158	0.3001	0.3889	0.3001
No log	9.4118	160	0.3015	0.3946	0.3015
No log	9.5294	162	0.3032	0.3946	0.3032
No log	9.6471	164	0.3050	0.3846	0.3050
No log	9.7647	166	0.3044	0.3732	0.3044
No log	9.8824	168	0.3039	0.3732	0.3039
No log	10.0	170	0.3037	0.3732	0.3037

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1