arabert_cross_vocabulary_task1_fold4

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3485
• Qwk: 0.8274
• Mse: 0.3485

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.125	2	3.3451	0.0078	3.3451
No log	0.25	4	1.7732	0.1006	1.7732
No log	0.375	6	0.9918	0.3397	0.9918
No log	0.5	8	1.0024	0.4328	1.0024
No log	0.625	10	0.7301	0.4216	0.7301
No log	0.75	12	0.5802	0.5631	0.5802
No log	0.875	14	0.6362	0.6894	0.6362
No log	1.0	16	0.7395	0.5514	0.7395
No log	1.125	18	0.6416	0.6725	0.6416
No log	1.25	20	0.4261	0.7695	0.4261
No log	1.375	22	0.4028	0.7538	0.4028
No log	1.5	24	0.4106	0.7411	0.4106
No log	1.625	26	0.4467	0.8262	0.4467
No log	1.75	28	0.5005	0.8064	0.5005
No log	1.875	30	0.4228	0.7273	0.4228
No log	2.0	32	0.4315	0.6938	0.4315
No log	2.125	34	0.5164	0.7418	0.5164
No log	2.25	36	0.5729	0.7839	0.5729
No log	2.375	38	0.4661	0.8238	0.4661
No log	2.5	40	0.3578	0.7987	0.3578
No log	2.625	42	0.3452	0.7759	0.3452
No log	2.75	44	0.3973	0.8179	0.3973
No log	2.875	46	0.4218	0.8271	0.4218
No log	3.0	48	0.4017	0.8252	0.4017
No log	3.125	50	0.3564	0.8053	0.3564
No log	3.25	52	0.3357	0.8030	0.3357
No log	3.375	54	0.3797	0.8226	0.3797
No log	3.5	56	0.3918	0.8236	0.3918
No log	3.625	58	0.3278	0.8144	0.3278
No log	3.75	60	0.3256	0.8144	0.3256
No log	3.875	62	0.3437	0.8131	0.3437
No log	4.0	64	0.3613	0.8186	0.3613
No log	4.125	66	0.3373	0.8172	0.3373
No log	4.25	68	0.3211	0.8140	0.3211
No log	4.375	70	0.3426	0.8269	0.3426
No log	4.5	72	0.3901	0.8236	0.3901
No log	4.625	74	0.3966	0.8286	0.3966
No log	4.75	76	0.3865	0.8324	0.3865
No log	4.875	78	0.4241	0.8202	0.4241
No log	5.0	80	0.4385	0.8219	0.4385
No log	5.125	82	0.3778	0.8292	0.3778
No log	5.25	84	0.3614	0.8202	0.3614
No log	5.375	86	0.3526	0.8104	0.3526
No log	5.5	88	0.3321	0.8030	0.3321
No log	5.625	90	0.3304	0.8053	0.3304
No log	5.75	92	0.3951	0.8300	0.3951
No log	5.875	94	0.4136	0.8290	0.4136
No log	6.0	96	0.3632	0.8271	0.3632
No log	6.125	98	0.3475	0.8169	0.3475
No log	6.25	100	0.3496	0.8214	0.3496
No log	6.375	102	0.3335	0.8234	0.3335
No log	6.5	104	0.3458	0.8254	0.3458
No log	6.625	106	0.3502	0.8263	0.3502
No log	6.75	108	0.3362	0.8182	0.3362
No log	6.875	110	0.3599	0.8388	0.3599
No log	7.0	112	0.4336	0.8458	0.4336
No log	7.125	114	0.5158	0.8403	0.5158
No log	7.25	116	0.4762	0.8348	0.4762
No log	7.375	118	0.3764	0.8311	0.3764
No log	7.5	120	0.3240	0.7922	0.3240
No log	7.625	122	0.3223	0.7975	0.3223
No log	7.75	124	0.3420	0.8307	0.3420
No log	7.875	126	0.4217	0.8418	0.4217
No log	8.0	128	0.5511	0.8211	0.5511
No log	8.125	130	0.6089	0.8259	0.6089
No log	8.25	132	0.5762	0.8177	0.5762
No log	8.375	134	0.4857	0.8304	0.4857
No log	8.5	136	0.3859	0.8416	0.3859
No log	8.625	138	0.3411	0.8297	0.3411
No log	8.75	140	0.3342	0.8297	0.3342
No log	8.875	142	0.3479	0.8274	0.3479
No log	9.0	144	0.3703	0.8408	0.3703
No log	9.125	146	0.3868	0.8480	0.3868
No log	9.25	148	0.3835	0.8520	0.3835
No log	9.375	150	0.3675	0.8384	0.3675
No log	9.5	152	0.3528	0.8250	0.3528
No log	9.625	154	0.3467	0.8274	0.3467
No log	9.75	156	0.3454	0.8274	0.3454
No log	9.875	158	0.3471	0.8274	0.3471
No log	10.0	160	0.3485	0.8274	0.3485

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1