arabert_cross_vocabulary_task7_fold6

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4590
• Qwk: 0.6447
• Mse: 0.4579

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0328	2	2.4310	0.0444	2.4182
No log	0.0656	4	1.0755	0.2942	1.0701
No log	0.0984	6	1.4087	0.2913	1.3952
No log	0.1311	8	1.4088	0.2931	1.3955
No log	0.1639	10	0.6928	0.4909	0.6873
No log	0.1967	12	0.5750	0.6609	0.5725
No log	0.2295	14	0.5006	0.6728	0.4989
No log	0.2623	16	0.4822	0.6065	0.4795
No log	0.2951	18	0.6545	0.5276	0.6499
No log	0.3279	20	1.0487	0.4061	1.0399
No log	0.3607	22	1.1184	0.3952	1.1077
No log	0.3934	24	1.0170	0.4299	1.0071
No log	0.4262	26	0.5774	0.5919	0.5737
No log	0.4590	28	0.4451	0.7458	0.4443
No log	0.4918	30	0.4414	0.7776	0.4412
No log	0.5246	32	0.4240	0.7423	0.4235
No log	0.5574	34	0.4446	0.6969	0.4439
No log	0.5902	36	0.6573	0.5608	0.6550
No log	0.6230	38	0.7881	0.4856	0.7846
No log	0.6557	40	0.7588	0.5118	0.7557
No log	0.6885	42	0.6330	0.5988	0.6310
No log	0.7213	44	0.5004	0.6314	0.4993
No log	0.7541	46	0.4319	0.6887	0.4312
No log	0.7869	48	0.4075	0.7028	0.4070
No log	0.8197	50	0.4010	0.7041	0.4005
No log	0.8525	52	0.3999	0.7089	0.3994
No log	0.8852	54	0.4118	0.6952	0.4111
No log	0.9180	56	0.4319	0.6733	0.4310
No log	0.9508	58	0.4484	0.6528	0.4474
No log	0.9836	60	0.4590	0.6447	0.4579

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1