arabert_baseline_development_task2_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5447
• Qwk: 0.4714
• Mse: 0.5518

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	3.5280	-0.0028	3.6293
No log	0.6667	4	1.1366	0.0251	1.1880
No log	1.0	6	0.4107	0.0	0.4259
No log	1.3333	8	0.4662	0.1765	0.4494
No log	1.6667	10	0.3079	0.2075	0.3114
No log	2.0	12	0.2932	0.4101	0.2962
No log	2.3333	14	0.3202	0.1525	0.3155
No log	2.6667	16	0.4604	0.2410	0.4445
No log	3.0	18	0.3972	0.1765	0.3866
No log	3.3333	20	0.2887	0.1356	0.2966
No log	3.6667	22	0.3203	0.1907	0.3343
No log	4.0	24	0.3286	0.3453	0.3381
No log	4.3333	26	0.4544	0.1674	0.4506
No log	4.6667	28	0.5203	0.1674	0.5108
No log	5.0	30	0.4560	0.4453	0.4535
No log	5.3333	32	0.4023	0.4457	0.4083
No log	5.6667	34	0.3835	0.3453	0.3956
No log	6.0	36	0.4165	0.4231	0.4249
No log	6.3333	38	0.5295	0.4152	0.5286
No log	6.6667	40	0.6129	0.2759	0.6063
No log	7.0	42	0.6169	0.2759	0.6109
No log	7.3333	44	0.5387	0.3971	0.5393
No log	7.6667	46	0.4830	0.3913	0.4895
No log	8.0	48	0.4890	0.3695	0.4962
No log	8.3333	50	0.5286	0.4714	0.5334
No log	8.6667	52	0.5656	0.3971	0.5690
No log	9.0	54	0.5673	0.4714	0.5716
No log	9.3333	56	0.5533	0.4714	0.5593
No log	9.6667	58	0.5477	0.4714	0.5545
No log	10.0	60	0.5447	0.4714	0.5518

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1