arabert_baseline_vocabulary_task7_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4434
• Qwk: 0.7306
• Mse: 0.4409

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.3333	2	1.1716	0.2597	1.1767
No log	0.6667	4	0.7370	0.5706	0.7641
No log	1.0	6	0.7497	0.5562	0.7703
No log	1.3333	8	0.6581	0.5987	0.6708
No log	1.6667	10	0.6557	0.4	0.6660
No log	2.0	12	0.4981	0.7242	0.5095
No log	2.3333	14	0.5966	0.5490	0.6027
No log	2.6667	16	0.7334	0.5273	0.7340
No log	3.0	18	0.7026	0.5268	0.7001
No log	3.3333	20	0.5537	0.6360	0.5517
No log	3.6667	22	0.5745	0.6402	0.5704
No log	4.0	24	0.6711	0.5418	0.6622
No log	4.3333	26	0.6481	0.5418	0.6370
No log	4.6667	28	0.5399	0.6015	0.5315
No log	5.0	30	0.4583	0.6890	0.4538
No log	5.3333	32	0.4300	0.7379	0.4272
No log	5.6667	34	0.4490	0.6402	0.4450
No log	6.0	36	0.5299	0.6360	0.5222
No log	6.3333	38	0.5460	0.5268	0.5387
No log	6.6667	40	0.4926	0.6360	0.4893
No log	7.0	42	0.4501	0.6402	0.4494
No log	7.3333	44	0.4428	0.7379	0.4421
No log	7.6667	46	0.4707	0.6360	0.4684
No log	8.0	48	0.4833	0.6360	0.4804
No log	8.3333	50	0.5033	0.6360	0.4988
No log	8.6667	52	0.5001	0.6360	0.4953
No log	9.0	54	0.4748	0.6833	0.4711
No log	9.3333	56	0.4541	0.6833	0.4512
No log	9.6667	58	0.4462	0.6833	0.4437
No log	10.0	60	0.4434	0.7306	0.4409

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1