arabert_baseline_grammar_task1_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0215
• Qwk: 0.4277
• Mse: 1.0149

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	6.3410	0.0064	6.3164
No log	0.6667	4	2.8727	-0.0032	2.8725
No log	1.0	6	1.9270	0.0637	1.9299
No log	1.3333	8	1.0990	0.0473	1.1001
No log	1.6667	10	0.8012	0.1845	0.8013
No log	2.0	12	1.1169	0.0073	1.1140
No log	2.3333	14	1.7714	0.0178	1.7742
No log	2.6667	16	1.2969	0.0732	1.2959
No log	3.0	18	0.7829	0.1901	0.7782
No log	3.3333	20	0.6867	0.2424	0.6831
No log	3.6667	22	0.7196	0.3083	0.7164
No log	4.0	24	0.9069	0.2976	0.9039
No log	4.3333	26	1.0224	0.2067	1.0193
No log	4.6667	28	1.1040	0.2031	1.1003
No log	5.0	30	0.9157	0.3099	0.9103
No log	5.3333	32	0.8876	0.3542	0.8806
No log	5.6667	34	1.0466	0.3092	1.0395
No log	6.0	36	1.1926	0.2415	1.1858
No log	6.3333	38	1.0798	0.2979	1.0719
No log	6.6667	40	0.9440	0.4539	0.9342
No log	7.0	42	0.9339	0.4539	0.9235
No log	7.3333	44	1.0280	0.2979	1.0189
No log	7.6667	46	1.1837	0.2266	1.1769
No log	8.0	48	1.2050	0.2266	1.1989
No log	8.3333	50	1.1372	0.3277	1.1307
No log	8.6667	52	1.0392	0.3092	1.0317
No log	9.0	54	0.9980	0.4027	0.9903
No log	9.3333	56	0.9952	0.4027	0.9879
No log	9.6667	58	1.0174	0.4277	1.0106
No log	10.0	60	1.0215	0.4277	1.0149

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1