arabert_baseline_style_task5_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6425
• Qwk: 0.6667
• Mse: 0.6425

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	1.7460	0.1985	1.7460
No log	0.6667	4	1.5870	0.0	1.5870
No log	1.0	6	1.4018	0.0	1.4018
No log	1.3333	8	1.2599	0.0	1.2599
No log	1.6667	10	1.2011	0.1294	1.2011
No log	2.0	12	1.1822	0.2077	1.1822
No log	2.3333	14	1.1032	0.3860	1.1032
No log	2.6667	16	1.0087	0.3860	1.0087
No log	3.0	18	0.9075	0.5652	0.9075
No log	3.3333	20	0.8395	0.4898	0.8395
No log	3.6667	22	0.7981	0.4898	0.7981
No log	4.0	24	0.7479	0.5455	0.7479
No log	4.3333	26	0.7093	0.5455	0.7093
No log	4.6667	28	0.6855	0.6222	0.6855
No log	5.0	30	0.6929	0.5614	0.6929
No log	5.3333	32	0.7079	0.4813	0.7079
No log	5.6667	34	0.6704	0.6222	0.6704
No log	6.0	36	0.6443	0.5946	0.6443
No log	6.3333	38	0.6496	0.5946	0.6496
No log	6.6667	40	0.6556	0.6119	0.6556
No log	7.0	42	0.6527	0.5946	0.6527
No log	7.3333	44	0.6454	0.5946	0.6454
No log	7.6667	46	0.6421	0.6186	0.6421
No log	8.0	48	0.6433	0.6186	0.6433
No log	8.3333	50	0.6419	0.6352	0.6419
No log	8.6667	52	0.6369	0.6352	0.6369
No log	9.0	54	0.6384	0.6667	0.6384
No log	9.3333	56	0.6431	0.6667	0.6431
No log	9.6667	58	0.6432	0.6667	0.6432
No log	10.0	60	0.6425	0.6667	0.6425

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1