arabert_cross_vocabulary_task6_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6146
• Qwk: 0.4250
• Mse: 0.6146

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0328	2	3.4679	0.0168	3.4679
No log	0.0656	4	1.6537	0.0514	1.6537
No log	0.0984	6	1.1313	0.1099	1.1313
No log	0.1311	8	0.9535	0.1805	0.9535
No log	0.1639	10	1.0794	0.3174	1.0794
No log	0.1967	12	0.6168	0.4395	0.6168
No log	0.2295	14	0.6033	0.4645	0.6033
No log	0.2623	16	0.6737	0.4499	0.6737
No log	0.2951	18	1.0668	0.3641	1.0668
No log	0.3279	20	1.3034	0.3203	1.3034
No log	0.3607	22	1.0248	0.3778	1.0248
No log	0.3934	24	0.6797	0.4447	0.6797
No log	0.4262	26	0.6365	0.4647	0.6365
No log	0.4590	28	0.7543	0.4079	0.7543
No log	0.4918	30	1.0103	0.3524	1.0103
No log	0.5246	32	0.8599	0.3860	0.8599
No log	0.5574	34	0.6772	0.4252	0.6772
No log	0.5902	36	0.5781	0.4282	0.5781
No log	0.6230	38	0.5748	0.4282	0.5748
No log	0.6557	40	0.6074	0.4195	0.6074
No log	0.6885	42	0.6402	0.3992	0.6402
No log	0.7213	44	0.7139	0.3932	0.7139
No log	0.7541	46	0.7877	0.3858	0.7877
No log	0.7869	48	0.7654	0.3896	0.7654
No log	0.8197	50	0.6949	0.4032	0.6949
No log	0.8525	52	0.6439	0.4209	0.6439
No log	0.8852	54	0.6372	0.4216	0.6372
No log	0.9180	56	0.6231	0.4191	0.6231
No log	0.9508	58	0.6184	0.4250	0.6184
No log	0.9836	60	0.6146	0.4250	0.6146

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1