arabert_baseline_development_task1_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5902
• Qwk: 0.5813
• Mse: 0.5907

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	4.3908	-0.0120	4.3758
No log	0.6667	4	1.6600	0.1250	1.6454
No log	1.0	6	0.7866	0.1857	0.7857
No log	1.3333	8	0.8027	0.3213	0.8042
No log	1.6667	10	0.7663	0.4436	0.7681
No log	2.0	12	0.6285	0.5382	0.6319
No log	2.3333	14	0.5709	0.5522	0.5784
No log	2.6667	16	0.6087	0.5551	0.6118
No log	3.0	18	0.6828	0.5551	0.6824
No log	3.3333	20	0.7735	0.5415	0.7740
No log	3.6667	22	0.6982	0.5682	0.6989
No log	4.0	24	0.7101	0.5682	0.7116
No log	4.3333	26	0.6932	0.5682	0.6950
No log	4.6667	28	0.6914	0.5682	0.6938
No log	5.0	30	0.6162	0.5477	0.6186
No log	5.3333	32	0.5655	0.5477	0.5678
No log	5.6667	34	0.5450	0.5445	0.5471
No log	6.0	36	0.5674	0.5477	0.5701
No log	6.3333	38	0.6194	0.5314	0.6234
No log	6.6667	40	0.5904	0.5477	0.5926
No log	7.0	42	0.5761	0.5445	0.5768
No log	7.3333	44	0.5947	0.5813	0.5951
No log	7.6667	46	0.6094	0.5825	0.6096
No log	8.0	48	0.6288	0.5995	0.6294
No log	8.3333	50	0.6317	0.5269	0.6325
No log	8.6667	52	0.6245	0.5995	0.6251
No log	9.0	54	0.6067	0.5825	0.6071
No log	9.3333	56	0.5975	0.5813	0.5979
No log	9.6667	58	0.5916	0.5813	0.5921
No log	10.0	60	0.5902	0.5813	0.5907

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1