arabert_baseline_grammar_task5_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4829
• Qwk: 0.6262
• Mse: 0.4829

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	2.4374	0.0250	2.4374
No log	0.6667	4	0.9732	-0.0090	0.9732
No log	1.0	6	0.5943	0.3165	0.5943
No log	1.3333	8	0.5108	0.3182	0.5108
No log	1.6667	10	0.4747	0.3137	0.4747
No log	2.0	12	0.4634	0.3165	0.4634
No log	2.3333	14	0.4798	0.4509	0.4798
No log	2.6667	16	0.4795	0.4737	0.4795
No log	3.0	18	0.5467	0.5327	0.5467
No log	3.3333	20	0.5831	0.5327	0.5831
No log	3.6667	22	0.5213	0.6269	0.5213
No log	4.0	24	0.6213	0.7087	0.6213
No log	4.3333	26	0.6774	0.7236	0.6774
No log	4.6667	28	0.6694	0.7236	0.6694
No log	5.0	30	0.5668	0.7	0.5668
No log	5.3333	32	0.5235	0.7059	0.5235
No log	5.6667	34	0.5216	0.7059	0.5216
No log	6.0	36	0.5070	0.5957	0.5070
No log	6.3333	38	0.5038	0.6047	0.5038
No log	6.6667	40	0.5220	0.6606	0.5220
No log	7.0	42	0.5420	0.6377	0.5420
No log	7.3333	44	0.5474	0.6667	0.5474
No log	7.6667	46	0.5400	0.6262	0.5400
No log	8.0	48	0.5341	0.6262	0.5341
No log	8.3333	50	0.5282	0.6262	0.5282
No log	8.6667	52	0.5146	0.6262	0.5146
No log	9.0	54	0.4982	0.6262	0.4982
No log	9.3333	56	0.4857	0.6262	0.4857
No log	9.6667	58	0.4846	0.6262	0.4846
No log	10.0	60	0.4829	0.6262	0.4829

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1