arabert_baseline_grammar_task5_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6836
• Qwk: 0.5644
• Mse: 0.6836

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	1.8956	0.1818	1.8956
No log	0.6667	4	1.7327	0.0	1.7327
No log	1.0	6	1.5736	0.0	1.5736
No log	1.3333	8	1.4541	0.0	1.4541
No log	1.6667	10	1.3999	0.1860	1.3999
No log	2.0	12	1.3873	0.2099	1.3873
No log	2.3333	14	1.2920	0.3077	1.2920
No log	2.6667	16	1.1687	0.3580	1.1687
No log	3.0	18	1.0389	0.4651	1.0389
No log	3.3333	20	0.9570	0.5435	0.9570
No log	3.6667	22	0.9091	0.5165	0.9091
No log	4.0	24	0.8399	0.5876	0.8399
No log	4.3333	26	0.7899	0.5876	0.7899
No log	4.6667	28	0.7622	0.5876	0.7622
No log	5.0	30	0.7525	0.5876	0.7525
No log	5.3333	32	0.7384	0.5876	0.7384
No log	5.6667	34	0.7273	0.5876	0.7273
No log	6.0	36	0.7207	0.5625	0.7207
No log	6.3333	38	0.7165	0.5625	0.7165
No log	6.6667	40	0.7152	0.5625	0.7152
No log	7.0	42	0.6996	0.5625	0.6996
No log	7.3333	44	0.6893	0.5876	0.6893
No log	7.6667	46	0.6890	0.5876	0.6890
No log	8.0	48	0.6887	0.5876	0.6887
No log	8.3333	50	0.6864	0.5625	0.6864
No log	8.6667	52	0.6829	0.6117	0.6829
No log	9.0	54	0.6833	0.5644	0.6833
No log	9.3333	56	0.6837	0.5644	0.6837
No log	9.6667	58	0.6836	0.5644	0.6836
No log	10.0	60	0.6836	0.5644	0.6836

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1