arabert_cross_relevance_task7_fold4

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3071
• Qwk: 0.5503
• Mse: 0.3071

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1333	2	0.5503	0.1601	0.5503
No log	0.2667	4	0.4709	0.1756	0.4709
No log	0.4	6	0.5461	0.3162	0.5461
No log	0.5333	8	0.4597	0.2317	0.4597
No log	0.6667	10	0.3774	0.2343	0.3774
No log	0.8	12	0.3353	0.2749	0.3353
No log	0.9333	14	0.3174	0.2948	0.3174
No log	1.0667	16	0.2902	0.3035	0.2902
No log	1.2	18	0.2766	0.3168	0.2766
No log	1.3333	20	0.2981	0.3060	0.2981
No log	1.4667	22	0.2863	0.3304	0.2863
No log	1.6	24	0.2902	0.3012	0.2902
No log	1.7333	26	0.2762	0.3012	0.2762
No log	1.8667	28	0.2673	0.3216	0.2673
No log	2.0	30	0.2662	0.3673	0.2662
No log	2.1333	32	0.2927	0.4416	0.2927
No log	2.2667	34	0.3407	0.4358	0.3407
No log	2.4	36	0.3384	0.4477	0.3384
No log	2.5333	38	0.3010	0.4244	0.3010
No log	2.6667	40	0.3009	0.3892	0.3009
No log	2.8	42	0.3374	0.4316	0.3374
No log	2.9333	44	0.3314	0.5561	0.3314
No log	3.0667	46	0.3496	0.6218	0.3496
No log	3.2	48	0.3271	0.5809	0.3271
No log	3.3333	50	0.2795	0.4989	0.2795
No log	3.4667	52	0.2819	0.4439	0.2819
No log	3.6	54	0.3087	0.3899	0.3087
No log	3.7333	56	0.3129	0.3657	0.3129
No log	3.8667	58	0.3247	0.4403	0.3247
No log	4.0	60	0.3497	0.5590	0.3497
No log	4.1333	62	0.3554	0.5878	0.3554
No log	4.2667	64	0.3325	0.5931	0.3325
No log	4.4	66	0.2900	0.5432	0.2900
No log	4.5333	68	0.2886	0.5107	0.2886
No log	4.6667	70	0.3060	0.4830	0.3060
No log	4.8	72	0.3388	0.4839	0.3388
No log	4.9333	74	0.3650	0.4978	0.3650
No log	5.0667	76	0.3668	0.5231	0.3668
No log	5.2	78	0.3779	0.5912	0.3779
No log	5.3333	80	0.3797	0.6118	0.3797
No log	5.4667	82	0.3500	0.5912	0.3500
No log	5.6	84	0.3089	0.5757	0.3089
No log	5.7333	86	0.2927	0.5560	0.2927
No log	5.8667	88	0.3153	0.5457	0.3153
No log	6.0	90	0.3342	0.5489	0.3342
No log	6.1333	92	0.3318	0.5833	0.3318
No log	6.2667	94	0.3329	0.5929	0.3329
No log	6.4	96	0.3261	0.5905	0.3261
No log	6.5333	98	0.3147	0.5945	0.3147
No log	6.6667	100	0.3034	0.5890	0.3034
No log	6.8	102	0.3084	0.5863	0.3084
No log	6.9333	104	0.3094	0.5743	0.3094
No log	7.0667	106	0.3099	0.5635	0.3099
No log	7.2	108	0.3101	0.5743	0.3101
No log	7.3333	110	0.3126	0.5689	0.3126
No log	7.4667	112	0.3295	0.5777	0.3295
No log	7.6	114	0.3516	0.5943	0.3516
No log	7.7333	116	0.3630	0.5902	0.3630
No log	7.8667	118	0.3567	0.5791	0.3567
No log	8.0	120	0.3425	0.5667	0.3425
No log	8.1333	122	0.3293	0.5424	0.3293
No log	8.2667	124	0.3261	0.5369	0.3261
No log	8.4	126	0.3225	0.5728	0.3225
No log	8.5333	128	0.3121	0.5644	0.3121
No log	8.6667	130	0.3072	0.5740	0.3072
No log	8.8	132	0.3079	0.5644	0.3079
No log	8.9333	134	0.3012	0.5687	0.3012
No log	9.0667	136	0.3021	0.5644	0.3021
No log	9.2	138	0.3058	0.5644	0.3058
No log	9.3333	140	0.3067	0.5557	0.3067
No log	9.4667	142	0.3080	0.5663	0.3080
No log	9.6	144	0.3079	0.5663	0.3079
No log	9.7333	146	0.3063	0.5493	0.3063
No log	9.8667	148	0.3065	0.5547	0.3065
No log	10.0	150	0.3071	0.5503	0.3071

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1