arabert_baseline_mechanics_task5_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6428
• Qwk: 0.8185
• Mse: 0.6428

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	1.2787	0.2092	1.2787
No log	0.6667	4	1.1600	0.0	1.1600
No log	1.0	6	1.0539	0.0	1.0539
No log	1.3333	8	0.9571	0.0	0.9571
No log	1.6667	10	0.9644	0.1447	0.9644
No log	2.0	12	0.9860	0.2857	0.9860
No log	2.3333	14	0.9563	0.3254	0.9563
No log	2.6667	16	0.8909	0.4531	0.8909
No log	3.0	18	0.8374	0.4531	0.8374
No log	3.3333	20	0.7882	0.5	0.7882
No log	3.6667	22	0.7735	0.5	0.7735
No log	4.0	24	0.7048	0.4932	0.7048
No log	4.3333	26	0.6450	0.4408	0.6450
No log	4.6667	28	0.5868	0.6307	0.5868
No log	5.0	30	0.5327	0.5588	0.5327
No log	5.3333	32	0.5046	0.6154	0.5046
No log	5.6667	34	0.4945	0.6154	0.4945
No log	6.0	36	0.5109	0.6277	0.5109
No log	6.3333	38	0.5298	0.6934	0.5298
No log	6.6667	40	0.5625	0.6875	0.5625
No log	7.0	42	0.5920	0.6759	0.5920
No log	7.3333	44	0.6178	0.7098	0.6178
No log	7.6667	46	0.6508	0.7845	0.6508
No log	8.0	48	0.6616	0.7845	0.6616
No log	8.3333	50	0.6750	0.7845	0.6750
No log	8.6667	52	0.6591	0.7881	0.6591
No log	9.0	54	0.6523	0.8185	0.6523
No log	9.3333	56	0.6489	0.8185	0.6489
No log	9.6667	58	0.6443	0.8185	0.6443
No log	10.0	60	0.6428	0.8185	0.6428

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1