arabert_baseline_style_task2_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2735
• Qwk: 0.5032
• Mse: 0.2833

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	4.3795	-0.0067	4.5325
No log	0.6667	4	1.5557	0.0644	1.6441
No log	1.0	6	0.6609	0.0	0.7022
No log	1.3333	8	0.5071	0.0	0.5017
No log	1.6667	10	0.5796	0.0	0.5647
No log	2.0	12	0.3894	0.4	0.4110
No log	2.3333	14	0.4086	0.2062	0.4404
No log	2.6667	16	0.3368	0.3186	0.3500
No log	3.0	18	0.3285	0.3269	0.3345
No log	3.3333	20	0.3077	0.3186	0.3155
No log	3.6667	22	0.2646	0.4262	0.2783
No log	4.0	24	0.2477	0.4	0.2670
No log	4.3333	26	0.2314	0.4	0.2483
No log	4.6667	28	0.2262	0.4122	0.2420
No log	5.0	30	0.2567	0.4425	0.2647
No log	5.3333	32	0.3341	0.3269	0.3334
No log	5.6667	34	0.2994	0.4425	0.3020
No log	6.0	36	0.2391	0.4	0.2498
No log	6.3333	38	0.2339	0.4	0.2445
No log	6.6667	40	0.2305	0.4	0.2408
No log	7.0	42	0.2435	0.4122	0.2512
No log	7.3333	44	0.2638	0.4122	0.2692
No log	7.6667	46	0.2765	0.4122	0.2815
No log	8.0	48	0.2797	0.4122	0.2854
No log	8.3333	50	0.2854	0.4539	0.2913
No log	8.6667	52	0.2874	0.5205	0.2938
No log	9.0	54	0.2791	0.475	0.2872
No log	9.3333	56	0.2716	0.5032	0.2813
No log	9.6667	58	0.2731	0.5032	0.2829
No log	10.0	60	0.2735	0.5032	0.2833

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1