arabert_cross_vocabulary_task4_fold5

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4552
• Qwk: 0.8443
• Mse: 0.4552

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0308	2	2.6954	0.0251	2.6954
No log	0.0615	4	1.4041	0.0989	1.4041
No log	0.0923	6	0.9765	0.4109	0.9765
No log	0.1231	8	0.8560	0.4734	0.8560
No log	0.1538	10	0.9060	0.7021	0.9060
No log	0.1846	12	1.1885	0.7065	1.1885
No log	0.2154	14	1.1782	0.6791	1.1782
No log	0.2462	16	0.7484	0.7647	0.7484
No log	0.2769	18	0.5200	0.6345	0.5200
No log	0.3077	20	0.5510	0.6112	0.5510
No log	0.3385	22	0.4929	0.6304	0.4929
No log	0.3692	24	0.4940	0.7245	0.4940
No log	0.4	26	0.5988	0.7499	0.5988
No log	0.4308	28	0.5987	0.7720	0.5987
No log	0.4615	30	0.4711	0.7843	0.4711
No log	0.4923	32	0.4119	0.8110	0.4119
No log	0.5231	34	0.3910	0.7703	0.3910
No log	0.5538	36	0.4012	0.7999	0.4012
No log	0.5846	38	0.4898	0.8328	0.4898
No log	0.6154	40	0.5931	0.8456	0.5931
No log	0.6462	42	0.5964	0.8475	0.5964
No log	0.6769	44	0.4983	0.8406	0.4983
No log	0.7077	46	0.3917	0.8332	0.3917
No log	0.7385	48	0.3517	0.8165	0.3517
No log	0.7692	50	0.3408	0.8087	0.3408
No log	0.8	52	0.3422	0.8165	0.3422
No log	0.8308	54	0.3585	0.8306	0.3585
No log	0.8615	56	0.3767	0.8335	0.3767
No log	0.8923	58	0.3910	0.8386	0.3910
No log	0.9231	60	0.4179	0.8461	0.4179
No log	0.9538	62	0.4428	0.8471	0.4428
No log	0.9846	64	0.4552	0.8443	0.4552

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1