arabert_cross_vocabulary_task6_fold5

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3785
• Qwk: 0.8516
• Mse: 0.3785

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0328	2	3.9811	0.0	3.9811
No log	0.0656	4	2.1407	0.1479	2.1407
No log	0.0984	6	1.1264	0.1813	1.1264
No log	0.1311	8	0.8366	0.5074	0.8366
No log	0.1639	10	0.8195	0.5441	0.8195
No log	0.1967	12	1.0058	0.5881	1.0058
No log	0.2295	14	1.0622	0.6346	1.0622
No log	0.2623	16	0.9868	0.6510	0.9868
No log	0.2951	18	0.6515	0.7729	0.6515
No log	0.3279	20	0.3945	0.6974	0.3945
No log	0.3607	22	0.5838	0.5376	0.5838
No log	0.3934	24	0.3960	0.6758	0.3960
No log	0.4262	26	0.3530	0.8548	0.3530
No log	0.4590	28	0.6972	0.8507	0.6972
No log	0.4918	30	0.8848	0.8086	0.8848
No log	0.5246	32	0.8411	0.8211	0.8411
No log	0.5574	34	0.7868	0.8137	0.7868
No log	0.5902	36	0.6804	0.8522	0.6804
No log	0.6230	38	0.5836	0.8642	0.5836
No log	0.6557	40	0.4481	0.8666	0.4481
No log	0.6885	42	0.3439	0.8485	0.3439
No log	0.7213	44	0.3168	0.8084	0.3168
No log	0.7541	46	0.3134	0.7968	0.3134
No log	0.7869	48	0.3070	0.7933	0.3070
No log	0.8197	50	0.3050	0.8144	0.3050
No log	0.8525	52	0.3093	0.8265	0.3093
No log	0.8852	54	0.3243	0.8430	0.3243
No log	0.9180	56	0.3471	0.8435	0.3471
No log	0.9508	58	0.3676	0.8531	0.3676
No log	0.9836	60	0.3785	0.8516	0.3785

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1