arabert_baseline_style_task2_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6287
• Qwk: 0.2903
• Mse: 0.6083

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	5.5617	-0.0100	5.5560
No log	0.6667	4	2.6178	0.0539	2.6238
No log	1.0	6	1.5516	0.0077	1.5429
No log	1.3333	8	0.8363	0.0192	0.8282
No log	1.6667	10	0.8891	0.0	0.8810
No log	2.0	12	1.0515	0.0	1.0369
No log	2.3333	14	1.0120	0.0187	0.9900
No log	2.6667	16	0.9023	0.0597	0.8789
No log	3.0	18	1.1300	0.1259	1.1065
No log	3.3333	20	1.0224	0.2296	0.9998
No log	3.6667	22	0.7522	0.3247	0.7319
No log	4.0	24	0.7846	0.125	0.7714
No log	4.3333	26	0.8034	0.2092	0.7904
No log	4.6667	28	0.7718	0.3046	0.7530
No log	5.0	30	0.7274	0.3046	0.7075
No log	5.3333	32	0.7003	0.3046	0.6801
No log	5.6667	34	0.6891	0.3046	0.6689
No log	6.0	36	0.6395	0.2025	0.6184
No log	6.3333	38	0.6234	0.2734	0.6015
No log	6.6667	40	0.6182	0.2734	0.5966
No log	7.0	42	0.6234	0.3383	0.6022
No log	7.3333	44	0.6278	0.3383	0.6070
No log	7.6667	46	0.6152	0.2785	0.5942
No log	8.0	48	0.6021	0.2785	0.5810
No log	8.3333	50	0.6118	0.2045	0.5911
No log	8.6667	52	0.6193	0.2903	0.5987
No log	9.0	54	0.6218	0.2903	0.6012
No log	9.3333	56	0.6223	0.2903	0.6017
No log	9.6667	58	0.6276	0.2903	0.6071
No log	10.0	60	0.6287	0.2903	0.6083

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1