arabert_baseline_vocabulary_task6_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6726
• Qwk: 0.5254
• Mse: 0.6726

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.5	2	2.0038	0.1716	2.0038
No log	1.0	4	1.1639	0.2784	1.1639
No log	1.5	6	1.1803	0.3913	1.1803
No log	2.0	8	1.1071	0.3913	1.1071
No log	2.5	10	0.9655	0.3704	0.9655
No log	3.0	12	0.8333	0.3913	0.8333
No log	3.5	14	0.7862	0.5842	0.7862
No log	4.0	16	0.8551	0.7177	0.8551
No log	4.5	18	0.8190	0.7287	0.8190
No log	5.0	20	0.7070	0.6667	0.7070
No log	5.5	22	0.7251	0.5776	0.7251
No log	6.0	24	0.6939	0.6667	0.6939
No log	6.5	26	0.6876	0.7083	0.6876
No log	7.0	28	0.7065	0.7829	0.7065
No log	7.5	30	0.7210	0.7829	0.7210
No log	8.0	32	0.6907	0.7470	0.6907
No log	8.5	34	0.6729	0.7083	0.6729
No log	9.0	36	0.6723	0.7083	0.6723
No log	9.5	38	0.6740	0.5254	0.6740
No log	10.0	40	0.6726	0.5254	0.6726

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1