arabert_baseline_grammar_task2_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5805
• Qwk: 0.4573
• Mse: 0.5856

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	4.5766	-0.0114	4.6757
No log	0.6667	4	1.8451	0.0840	1.9011
No log	1.0	6	1.0385	0.0	1.0614
No log	1.3333	8	0.8509	0.0	0.8474
No log	1.6667	10	0.8477	0.0	0.8402
No log	2.0	12	0.7153	0.2184	0.7165
No log	2.3333	14	0.6748	0.2687	0.6856
No log	2.6667	16	0.7113	0.1912	0.7272
No log	3.0	18	0.7379	0.1902	0.7546
No log	3.3333	20	0.6880	0.1923	0.7014
No log	3.6667	22	0.6478	0.3544	0.6544
No log	4.0	24	0.6652	0.3849	0.6682
No log	4.3333	26	0.6717	0.4573	0.6748
No log	4.6667	28	0.6419	0.4304	0.6475
No log	5.0	30	0.6314	0.4573	0.6355
No log	5.3333	32	0.6776	0.3849	0.6787
No log	5.6667	34	0.6654	0.3849	0.6671
No log	6.0	36	0.6226	0.4573	0.6260
No log	6.3333	38	0.5977	0.3544	0.6024
No log	6.6667	40	0.5953	0.1306	0.6010
No log	7.0	42	0.5909	0.1306	0.5966
No log	7.3333	44	0.5860	0.1306	0.5916
No log	7.6667	46	0.5891	0.3544	0.5944
No log	8.0	48	0.6067	0.4573	0.6111
No log	8.3333	50	0.6190	0.4573	0.6228
No log	8.6667	52	0.6206	0.4573	0.6243
No log	9.0	54	0.6083	0.4573	0.6124
No log	9.3333	56	0.5947	0.4573	0.5993
No log	9.6667	58	0.5850	0.4573	0.5899
No log	10.0	60	0.5805	0.4573	0.5856

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1