arabert_baseline_grammar_task1_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5311
• Qwk: 0.5882
• Mse: 0.5332

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	4.8751	-0.0497	4.7777
No log	0.6667	4	1.9817	0.0263	1.9110
No log	1.0	6	1.0160	0.1348	0.9761
No log	1.3333	8	0.8888	0.2611	0.8614
No log	1.6667	10	0.7375	0.4340	0.7296
No log	2.0	12	0.9526	0.2222	0.9430
No log	2.3333	14	0.8202	0.3636	0.8111
No log	2.6667	16	0.6506	0.4324	0.6405
No log	3.0	18	0.7279	0.4290	0.7066
No log	3.3333	20	0.8689	0.3668	0.8447
No log	3.6667	22	0.7419	0.4119	0.7226
No log	4.0	24	0.6583	0.5051	0.6408
No log	4.3333	26	0.6672	0.4726	0.6487
No log	4.6667	28	0.6625	0.4373	0.6466
No log	5.0	30	0.6519	0.4061	0.6397
No log	5.3333	32	0.6430	0.4358	0.6321
No log	5.6667	34	0.6395	0.4358	0.6295
No log	6.0	36	0.6008	0.5051	0.5923
No log	6.3333	38	0.5776	0.5051	0.5691
No log	6.6667	40	0.5657	0.5051	0.5566
No log	7.0	42	0.5537	0.5051	0.5464
No log	7.3333	44	0.5474	0.5051	0.5417
No log	7.6667	46	0.5355	0.5051	0.5321
No log	8.0	48	0.5261	0.5051	0.5245
No log	8.3333	50	0.5235	0.5882	0.5233
No log	8.6667	52	0.5271	0.5882	0.5280
No log	9.0	54	0.5282	0.5882	0.5300
No log	9.3333	56	0.5296	0.5882	0.5318
No log	9.6667	58	0.5304	0.5882	0.5327
No log	10.0	60	0.5311	0.5882	0.5332

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1