arabert_cross_vocabulary_task3_fold3

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8364
• Qwk: 0.7896
• Mse: 0.8364

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0317	2	2.7496	0.0472	2.7497
No log	0.0635	4	1.9727	0.1300	1.9727
No log	0.0952	6	1.5014	0.3559	1.5014
No log	0.1270	8	1.6487	0.3589	1.6487
No log	0.1587	10	1.5137	0.4770	1.5137
No log	0.1905	12	1.4075	0.3759	1.4075
No log	0.2222	14	1.2715	0.4953	1.2715
No log	0.2540	16	1.2869	0.6671	1.2869
No log	0.2857	18	1.2721	0.6981	1.2721
No log	0.3175	20	1.0831	0.7193	1.0831
No log	0.3492	22	0.9138	0.6190	0.9138
No log	0.3810	24	0.8698	0.6092	0.8698
No log	0.4127	26	0.8440	0.6788	0.8440
No log	0.4444	28	0.8595	0.7133	0.8595
No log	0.4762	30	0.9479	0.7455	0.9479
No log	0.5079	32	0.9721	0.7436	0.9721
No log	0.5397	34	0.8831	0.7446	0.8831
No log	0.5714	36	0.8298	0.7517	0.8298
No log	0.6032	38	0.8033	0.7530	0.8033
No log	0.6349	40	0.7634	0.7328	0.7634
No log	0.6667	42	0.7751	0.7525	0.7751
No log	0.6984	44	0.7763	0.7676	0.7763
No log	0.7302	46	0.7904	0.7742	0.7904
No log	0.7619	48	0.7987	0.7729	0.7987
No log	0.7937	50	0.7994	0.7773	0.7994
No log	0.8254	52	0.7974	0.7849	0.7974
No log	0.8571	54	0.8154	0.7849	0.8154
No log	0.8889	56	0.8322	0.7856	0.8322
No log	0.9206	58	0.8311	0.7868	0.8311
No log	0.9524	60	0.8346	0.7849	0.8346
No log	0.9841	62	0.8364	0.7896	0.8364

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1