arabert_baseline_vocabulary_task1_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4495
• Qwk: 0.5070
• Mse: 0.4537

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	6.1233	-0.0049	6.0762
No log	0.6667	4	2.5238	0.0438	2.5221
No log	1.0	6	1.6945	0.0677	1.7016
No log	1.3333	8	0.6878	0.2489	0.6921
No log	1.6667	10	0.5146	0.2759	0.5204
No log	2.0	12	0.6279	0.2036	0.6327
No log	2.3333	14	1.1313	0.0888	1.1412
No log	2.6667	16	0.8112	0.1508	0.8178
No log	3.0	18	0.5968	0.3475	0.6025
No log	3.3333	20	0.5587	0.3475	0.5649
No log	3.6667	22	0.3908	0.4828	0.3937
No log	4.0	24	0.3808	0.4828	0.3836
No log	4.3333	26	0.5135	0.4190	0.5196
No log	4.6667	28	0.7114	0.2825	0.7220
No log	5.0	30	0.6755	0.3456	0.6851
No log	5.3333	32	0.4917	0.5259	0.4969
No log	5.6667	34	0.4653	0.5679	0.4693
No log	6.0	36	0.4650	0.5917	0.4695
No log	6.3333	38	0.4511	0.5817	0.4552
No log	6.6667	40	0.4643	0.4296	0.4687
No log	7.0	42	0.4888	0.3970	0.4933
No log	7.3333	44	0.4987	0.3970	0.5033
No log	7.6667	46	0.4509	0.5191	0.4546
No log	8.0	48	0.3997	0.5917	0.4021
No log	8.3333	50	0.4026	0.5917	0.4048
No log	8.6667	52	0.4048	0.5817	0.4074
No log	9.0	54	0.4217	0.5366	0.4250
No log	9.3333	56	0.4348	0.5366	0.4386
No log	9.6667	58	0.4485	0.5070	0.4527
No log	10.0	60	0.4495	0.5070	0.4537

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1