arabert_cross_relevance_task1_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2738
• Qwk: 0.0
• Mse: 0.2739

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.125	2	1.0588	-0.0001	1.0575
No log	0.25	4	0.3264	0.1085	0.3266
No log	0.375	6	0.4800	0.0711	0.4804
No log	0.5	8	0.3741	0.0242	0.3744
No log	0.625	10	0.2864	0.0	0.2865
No log	0.75	12	0.2902	0.0	0.2903
No log	0.875	14	0.3836	-0.0180	0.3840
No log	1.0	16	0.5708	0.0324	0.5714
No log	1.125	18	0.6465	0.0638	0.6472
No log	1.25	20	0.4891	-0.0969	0.4897
No log	1.375	22	0.3947	0.0283	0.3952
No log	1.5	24	0.3523	0.0122	0.3527
No log	1.625	26	0.3225	0.0	0.3228
No log	1.75	28	0.3334	0.0	0.3337
No log	1.875	30	0.3339	0.0	0.3343
No log	2.0	32	0.3405	0.0122	0.3408
No log	2.125	34	0.3476	0.0122	0.3480
No log	2.25	36	0.3276	0.0122	0.3280
No log	2.375	38	0.3178	0.0	0.3181
No log	2.5	40	0.3023	0.0	0.3026
No log	2.625	42	0.2905	0.0	0.2907
No log	2.75	44	0.2841	0.0	0.2843
No log	2.875	46	0.2902	0.0	0.2904
No log	3.0	48	0.3167	0.0122	0.3170
No log	3.125	50	0.3680	0.0285	0.3684
No log	3.25	52	0.3771	0.0452	0.3775
No log	3.375	54	0.3850	0.0665	0.3854
No log	3.5	56	0.3485	0.0080	0.3489
No log	3.625	58	0.3149	0.0	0.3151
No log	3.75	60	0.2939	0.0	0.2941
No log	3.875	62	0.2881	0.0	0.2883
No log	4.0	64	0.2895	0.0	0.2897
No log	4.125	66	0.3127	0.0	0.3129
No log	4.25	68	0.3458	0.0245	0.3462
No log	4.375	70	0.3576	0.0161	0.3580
No log	4.5	72	0.3521	0.0161	0.3525
No log	4.625	74	0.3633	0.0161	0.3637
No log	4.75	76	0.3571	0.0326	0.3575
No log	4.875	78	0.3220	0.0	0.3223
No log	5.0	80	0.2971	0.0	0.2973
No log	5.125	82	0.2905	0.0	0.2906
No log	5.25	84	0.2904	0.0	0.2906
No log	5.375	86	0.2948	0.0	0.2950
No log	5.5	88	0.3083	0.0	0.3085
No log	5.625	90	0.3120	0.0	0.3123
No log	5.75	92	0.2947	0.0	0.2949
No log	5.875	94	0.2786	0.0	0.2786
No log	6.0	96	0.2717	0.0	0.2717
No log	6.125	98	0.2685	0.0	0.2684
No log	6.25	100	0.2677	0.0	0.2677
No log	6.375	102	0.2687	0.0	0.2688
No log	6.5	104	0.2689	0.0	0.2690
No log	6.625	106	0.2694	0.0	0.2695
No log	6.75	108	0.2703	0.0	0.2703
No log	6.875	110	0.2742	0.0	0.2743
No log	7.0	112	0.2832	0.0	0.2833
No log	7.125	114	0.2950	0.0	0.2953
No log	7.25	116	0.2962	0.0	0.2965
No log	7.375	118	0.2908	0.0	0.2910
No log	7.5	120	0.2842	0.0	0.2844
No log	7.625	122	0.2800	0.0	0.2802
No log	7.75	124	0.2757	0.0	0.2758
No log	7.875	126	0.2725	0.0	0.2726
No log	8.0	128	0.2720	0.0	0.2720
No log	8.125	130	0.2730	0.0	0.2730
No log	8.25	132	0.2749	0.0	0.2750
No log	8.375	134	0.2761	0.0	0.2763
No log	8.5	136	0.2761	0.0	0.2762
No log	8.625	138	0.2753	0.0	0.2755
No log	8.75	140	0.2739	0.0	0.2740
No log	8.875	142	0.2738	0.0	0.2739
No log	9.0	144	0.2744	0.0	0.2745
No log	9.125	146	0.2747	0.0	0.2748
No log	9.25	148	0.2751	0.0	0.2752
No log	9.375	150	0.2750	0.0	0.2751
No log	9.5	152	0.2745	0.0	0.2747
No log	9.625	154	0.2742	0.0	0.2743
No log	9.75	156	0.2740	0.0	0.2741
No log	9.875	158	0.2739	0.0	0.2740
No log	10.0	160	0.2738	0.0	0.2739

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1