arabert_cross_vocabulary_task3_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5285
• Qwk: 0.4405
• Mse: 0.5285

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0317	2	8.4052	0.0	8.4052
No log	0.0635	4	5.1861	-0.0063	5.1861
No log	0.0952	6	2.9254	0.0185	2.9254
No log	0.1270	8	1.8370	0.0864	1.8370
No log	0.1587	10	1.1161	0.0869	1.1161
No log	0.1905	12	0.8855	0.1061	0.8855
No log	0.2222	14	1.2541	0.1523	1.2541
No log	0.2540	16	1.9970	0.1566	1.9970
No log	0.2857	18	1.0435	0.2378	1.0435
No log	0.3175	20	0.6890	0.3618	0.6890
No log	0.3492	22	0.5804	0.4399	0.5804
No log	0.3810	24	0.5528	0.4709	0.5528
No log	0.4127	26	0.5886	0.4445	0.5886
No log	0.4444	28	0.8824	0.37	0.8824
No log	0.4762	30	0.8947	0.3615	0.8947
No log	0.5079	32	0.6729	0.4052	0.6729
No log	0.5397	34	0.5355	0.4536	0.5355
No log	0.5714	36	0.4786	0.4846	0.4786
No log	0.6032	38	0.4704	0.4846	0.4704
No log	0.6349	40	0.4945	0.4637	0.4945
No log	0.6667	42	0.6100	0.4202	0.6100
No log	0.6984	44	0.6733	0.3645	0.6733
No log	0.7302	46	0.6704	0.3694	0.6704
No log	0.7619	48	0.6390	0.3820	0.6390
No log	0.7937	50	0.5726	0.4157	0.5726
No log	0.8254	52	0.5435	0.4449	0.5435
No log	0.8571	54	0.5097	0.4543	0.5097
No log	0.8889	56	0.5010	0.4503	0.5010
No log	0.9206	58	0.5122	0.4418	0.5122
No log	0.9524	60	0.5275	0.4351	0.5275
No log	0.9841	62	0.5285	0.4405	0.5285

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1