arabert_baseline_development_task1_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5458
• Qwk: 0.4658
• Mse: 0.5512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	5.2478	-0.0019	5.2043
No log	0.6667	4	2.1589	0.0283	2.1601
No log	1.0	6	1.4355	0.0541	1.4399
No log	1.3333	8	0.6301	0.2222	0.6340
No log	1.6667	10	0.6183	0.1498	0.6241
No log	2.0	12	1.0561	0.0474	1.0672
No log	2.3333	14	1.4163	0.0173	1.4254
No log	2.6667	16	0.7226	0.1868	0.7270
No log	3.0	18	0.4156	0.3931	0.4169
No log	3.3333	20	0.3843	0.3824	0.3843
No log	3.6667	22	0.4180	0.3265	0.4208
No log	4.0	24	0.6135	0.2391	0.6210
No log	4.3333	26	0.6975	0.3265	0.7080
No log	4.6667	28	0.4353	0.3666	0.4431
No log	5.0	30	0.3291	0.5318	0.3307
No log	5.3333	32	0.3685	0.6995	0.3686
No log	5.6667	34	0.4312	0.5092	0.4349
No log	6.0	36	0.7352	0.3824	0.7472
No log	6.3333	38	0.8022	0.3824	0.8152
No log	6.6667	40	0.6388	0.3988	0.6477
No log	7.0	42	0.5174	0.4043	0.5223
No log	7.3333	44	0.4579	0.4988	0.4598
No log	7.6667	46	0.4255	0.6171	0.4251
No log	8.0	48	0.4172	0.6171	0.4166
No log	8.3333	50	0.4446	0.4989	0.4459
No log	8.6667	52	0.5029	0.5155	0.5068
No log	9.0	54	0.5447	0.4658	0.5502
No log	9.3333	56	0.5508	0.4658	0.5564
No log	9.6667	58	0.5519	0.4658	0.5575
No log	10.0	60	0.5458	0.4658	0.5512

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1