arabert_cross_vocabulary_task2_fold2

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7896
• Qwk: -0.0578
• Mse: 0.7896

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0308	2	8.5786	0.0	8.5786
No log	0.0615	4	4.7361	0.0027	4.7361
No log	0.0923	6	2.4849	-0.0164	2.4849
No log	0.1231	8	1.0682	0.0	1.0682
No log	0.1538	10	1.0484	-0.2071	1.0484
No log	0.1846	12	1.1580	-0.1541	1.1580
No log	0.2154	14	0.7197	0.0071	0.7197
No log	0.2462	16	0.7275	0.1341	0.7275
No log	0.2769	18	0.8223	0.0676	0.8223
No log	0.3077	20	0.8068	0.0447	0.8068
No log	0.3385	22	0.7532	-0.0157	0.7532
No log	0.3692	24	0.7525	-0.0155	0.7525
No log	0.4	26	0.7808	-0.0217	0.7808
No log	0.4308	28	0.7874	-0.0658	0.7874
No log	0.4615	30	0.7780	0.0	0.7780
No log	0.4923	32	0.8462	0.0	0.8462
No log	0.5231	34	0.9212	0.0	0.9212
No log	0.5538	36	0.9624	0.0	0.9624
No log	0.5846	38	0.9977	0.0	0.9977
No log	0.6154	40	0.9965	0.0	0.9965
No log	0.6462	42	0.9605	0.0	0.9605
No log	0.6769	44	0.8767	0.0	0.8767
No log	0.7077	46	0.8153	0.0	0.8153
No log	0.7385	48	0.7919	0.0	0.7919
No log	0.7692	50	0.7850	-0.0371	0.7850
No log	0.8	52	0.7871	-0.0777	0.7871
No log	0.8308	54	0.7856	-0.0777	0.7856
No log	0.8615	56	0.7873	-0.0777	0.7873
No log	0.8923	58	0.7877	-0.0578	0.7877
No log	0.9231	60	0.7921	-0.0777	0.7921
No log	0.9538	62	0.7919	-0.0777	0.7919
No log	0.9846	64	0.7896	-0.0578	0.7896

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1