README.md

metadata

base_model: aubmindlab/bert-base-arabertv02
tags:
  - generated_from_trainer
model-index:
  - name: arabert_cross_relevance_task1_fold6
    results: []

arabert_cross_relevance_task1_fold6

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3732
• Qwk: 0.2161
• Mse: 0.3733

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1333	2	0.9235	0.0305	0.9241
No log	0.2667	4	0.4213	0.0752	0.4205
No log	0.4	6	0.4900	0.2872	0.4904
No log	0.5333	8	0.3989	0.1492	0.3997
No log	0.6667	10	0.2875	0.0828	0.2881
No log	0.8	12	0.3103	0.1946	0.3107
No log	0.9333	14	0.3505	0.1869	0.3507
No log	1.0667	16	0.2987	0.2328	0.2991
No log	1.2	18	0.2743	0.1987	0.2749
No log	1.3333	20	0.2678	0.1801	0.2685
No log	1.4667	22	0.2718	0.2380	0.2723
No log	1.6	24	0.2750	0.2484	0.2754
No log	1.7333	26	0.2802	0.2407	0.2807
No log	1.8667	28	0.2967	0.2647	0.2971
No log	2.0	30	0.3002	0.2386	0.3005
No log	2.1333	32	0.3042	0.2307	0.3046
No log	2.2667	34	0.2836	0.2266	0.2840
No log	2.4	36	0.2566	0.2275	0.2571
No log	2.5333	38	0.2566	0.2726	0.2572
No log	2.6667	40	0.2685	0.2318	0.2689
No log	2.8	42	0.3239	0.2291	0.3238
No log	2.9333	44	0.3137	0.2140	0.3139
No log	3.0667	46	0.2868	0.2462	0.2874
No log	3.2	48	0.2840	0.2475	0.2845
No log	3.3333	50	0.2972	0.2539	0.2975
No log	3.4667	52	0.2987	0.2373	0.2989
No log	3.6	54	0.2922	0.2472	0.2925
No log	3.7333	56	0.2979	0.2276	0.2983
No log	3.8667	58	0.3082	0.2134	0.3086
No log	4.0	60	0.3245	0.2197	0.3249
No log	4.1333	62	0.3302	0.2197	0.3305
No log	4.2667	64	0.3164	0.2079	0.3169
No log	4.4	66	0.3410	0.2131	0.3413
No log	4.5333	68	0.3844	0.2178	0.3845
No log	4.6667	70	0.3887	0.2178	0.3888
No log	4.8	72	0.3995	0.2096	0.3996
No log	4.9333	74	0.3677	0.2141	0.3679
No log	5.0667	76	0.3517	0.2188	0.3520
No log	5.2	78	0.3437	0.2188	0.3439
No log	5.3333	80	0.3781	0.2096	0.3782
No log	5.4667	82	0.4079	0.1975	0.4080
No log	5.6	84	0.4051	0.2133	0.4052
No log	5.7333	86	0.3469	0.2313	0.3471
No log	5.8667	88	0.3091	0.2317	0.3095
No log	6.0	90	0.3034	0.2341	0.3039
No log	6.1333	92	0.3187	0.2175	0.3191
No log	6.2667	94	0.3556	0.2123	0.3558
No log	6.4	96	0.4256	0.1936	0.4256
No log	6.5333	98	0.4370	0.1899	0.4370
No log	6.6667	100	0.4141	0.2053	0.4142
No log	6.8	102	0.4062	0.2053	0.4063
No log	6.9333	104	0.3722	0.2178	0.3723
No log	7.0667	106	0.3420	0.2313	0.3423
No log	7.2	108	0.3323	0.2337	0.3326
No log	7.3333	110	0.3482	0.2161	0.3484
No log	7.4667	112	0.3690	0.2178	0.3691
No log	7.6	114	0.3867	0.2169	0.3868
No log	7.7333	116	0.4039	0.2029	0.4039
No log	7.8667	118	0.4092	0.2029	0.4091
No log	8.0	120	0.4179	0.2012	0.4178
No log	8.1333	122	0.4160	0.2089	0.4160
No log	8.2667	124	0.4199	0.2012	0.4199
No log	8.4	126	0.4338	0.1899	0.4337
No log	8.5333	128	0.4437	0.1935	0.4436
No log	8.6667	130	0.4404	0.1973	0.4403
No log	8.8	132	0.4283	0.2012	0.4282
No log	8.9333	134	0.4152	0.2012	0.4151
No log	9.0667	136	0.3937	0.2089	0.3937
No log	9.2	138	0.3753	0.2141	0.3753
No log	9.3333	140	0.3664	0.2097	0.3665
No log	9.4667	142	0.3644	0.2097	0.3645
No log	9.6	144	0.3651	0.2097	0.3652
No log	9.7333	146	0.3680	0.2097	0.3681
No log	9.8667	148	0.3713	0.2097	0.3714
No log	10.0	150	0.3732	0.2161	0.3733

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1