arabert_baseline_grammar_task7_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5572
• Qwk: 0.5704
• Mse: 0.5572

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	1.4661	0.0977	1.4661
No log	0.6667	4	0.7644	0.4444	0.7644
No log	1.0	6	0.8832	0.4242	0.8832
No log	1.3333	8	1.3294	0.4231	1.3294
No log	1.6667	10	0.7718	0.4272	0.7718
No log	2.0	12	0.4503	0.5679	0.4503
No log	2.3333	14	0.5309	0.5417	0.5309
No log	2.6667	16	0.5822	0.5149	0.5822
No log	3.0	18	0.9504	0.4242	0.9504
No log	3.3333	20	1.2071	0.4068	1.2071
No log	3.6667	22	0.8892	0.4242	0.8892
No log	4.0	24	0.5812	0.5149	0.5812
No log	4.3333	26	0.5065	0.5149	0.5065
No log	4.6667	28	0.5216	0.5149	0.5216
No log	5.0	30	0.5606	0.5091	0.5606
No log	5.3333	32	0.6888	0.4257	0.6888
No log	5.6667	34	0.6926	0.4257	0.6926
No log	6.0	36	0.5721	0.5139	0.5721
No log	6.3333	38	0.4835	0.6127	0.4835
No log	6.6667	40	0.4666	0.6127	0.4666
No log	7.0	42	0.4944	0.6127	0.4944
No log	7.3333	44	0.5939	0.5704	0.5939
No log	7.6667	46	0.7085	0.4828	0.7085
No log	8.0	48	0.7680	0.4620	0.7680
No log	8.3333	50	0.7277	0.4828	0.7277
No log	8.6667	52	0.6480	0.4828	0.6480
No log	9.0	54	0.6116	0.5085	0.6116
No log	9.3333	56	0.5764	0.5704	0.5764
No log	9.6667	58	0.5647	0.5704	0.5647
No log	10.0	60	0.5572	0.5704	0.5572

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1