arabert_cross_vocabulary_task2_fold6

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6523
• Qwk: 0.5530
• Mse: 0.6489

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0328	2	1.8840	0.0145	1.8809
No log	0.0656	4	0.9359	0.2886	0.9294
No log	0.0984	6	0.7301	0.4326	0.7244
No log	0.1311	8	0.8788	0.4198	0.8649
No log	0.1639	10	2.1531	0.2129	2.1311
No log	0.1967	12	1.2200	0.3928	1.2071
No log	0.2295	14	0.4403	0.6916	0.4393
No log	0.2623	16	0.3920	0.7036	0.3922
No log	0.2951	18	0.5236	0.5366	0.5237
No log	0.3279	20	0.9738	0.4072	0.9738
No log	0.3607	22	1.2122	0.3697	1.2128
No log	0.3934	24	0.9504	0.4239	0.9509
No log	0.4262	26	0.7372	0.4925	0.7381
No log	0.4590	28	0.6200	0.5858	0.6206
No log	0.4918	30	0.5103	0.6451	0.5105
No log	0.5246	32	0.4884	0.6459	0.4883
No log	0.5574	34	0.5462	0.5996	0.5459
No log	0.5902	36	0.6834	0.5246	0.6825
No log	0.6230	38	0.7927	0.4864	0.7913
No log	0.6557	40	0.7801	0.4921	0.7782
No log	0.6885	42	0.6863	0.5236	0.6844
No log	0.7213	44	0.6421	0.5737	0.6400
No log	0.7541	46	0.5559	0.6240	0.5539
No log	0.7869	48	0.5296	0.6240	0.5276
No log	0.8197	50	0.5617	0.6101	0.5592
No log	0.8525	52	0.6106	0.5850	0.6076
No log	0.8852	54	0.6289	0.5711	0.6257
No log	0.9180	56	0.6469	0.5601	0.6435
No log	0.9508	58	0.6495	0.5530	0.6461
No log	0.9836	60	0.6523	0.5530	0.6489

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1