arabert_baseline_relevance_task5_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3678
• Qwk: 0.2949
• Mse: 0.3678

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	0.4684	0.1304	0.4684
No log	0.6667	4	0.4252	0.0	0.4252
No log	1.0	6	0.4179	0.0	0.4179
No log	1.3333	8	0.4231	0.0	0.4231
No log	1.6667	10	0.4130	0.0	0.4130
No log	2.0	12	0.4098	0.0	0.4098
No log	2.3333	14	0.4153	0.0551	0.4153
No log	2.6667	16	0.4026	0.0551	0.4026
No log	3.0	18	0.3622	0.1837	0.3622
No log	3.3333	20	0.3561	0.1837	0.3561
No log	3.6667	22	0.3628	0.1837	0.3628
No log	4.0	24	0.3773	0.1837	0.3773
No log	4.3333	26	0.3744	0.1837	0.3744
No log	4.6667	28	0.3673	0.1837	0.3673
No log	5.0	30	0.3744	0.1837	0.3744
No log	5.3333	32	0.3730	0.1837	0.3730
No log	5.6667	34	0.3589	0.1333	0.3589
No log	6.0	36	0.3503	0.3464	0.3503
No log	6.3333	38	0.3556	0.2949	0.3556
No log	6.6667	40	0.3713	0.2949	0.3713
No log	7.0	42	0.3743	0.2949	0.3743
No log	7.3333	44	0.3742	0.2949	0.3742
No log	7.6667	46	0.3719	0.2949	0.3719
No log	8.0	48	0.3699	0.2949	0.3699
No log	8.3333	50	0.3712	0.2949	0.3712
No log	8.6667	52	0.3661	0.2949	0.3661
No log	9.0	54	0.3649	0.2157	0.3649
No log	9.3333	56	0.3664	0.1333	0.3664
No log	9.6667	58	0.3674	0.2157	0.3674
No log	10.0	60	0.3678	0.2949	0.3678

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1