arabert_cross_vocabulary_task6_fold2

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7242
• Qwk: 0.0
• Mse: 0.7136

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0299	2	3.2898	-0.0072	3.3109
No log	0.0597	4	1.5499	0.0064	1.5580
No log	0.0896	6	0.8567	-0.0149	0.8591
No log	0.1194	8	0.5737	-0.0086	0.5725
No log	0.1493	10	0.5552	-0.0146	0.5507
No log	0.1791	12	0.5914	-0.0252	0.5849
No log	0.2090	14	0.5900	0.0	0.5831
No log	0.2388	16	0.5625	0.0144	0.5568
No log	0.2687	18	0.5553	0.0188	0.5497
No log	0.2985	20	0.5729	-0.0264	0.5693
No log	0.3284	22	0.5862	-0.0635	0.5833
No log	0.3582	24	0.5679	0.0441	0.5643
No log	0.3881	26	0.5625	0.0732	0.5588
No log	0.4179	28	0.5473	0.0472	0.5424
No log	0.4478	30	0.5608	0.0	0.5545
No log	0.4776	32	0.6062	0.0	0.5987
No log	0.5075	34	0.6492	0.0	0.6414
No log	0.5373	36	0.6805	0.0	0.6725
No log	0.5672	38	0.7145	0.0	0.7051
No log	0.5970	40	0.7050	0.0	0.6949
No log	0.6269	42	0.6899	0.0	0.6796
No log	0.6567	44	0.7091	0.0	0.6986
No log	0.6866	46	0.7298	0.0	0.7190
No log	0.7164	48	0.7558	0.0	0.7450
No log	0.7463	50	0.7596	0.0	0.7490
No log	0.7761	52	0.7517	0.0	0.7414
No log	0.8060	54	0.7385	0.0	0.7283
No log	0.8358	56	0.7315	0.0	0.7211
No log	0.8657	58	0.7354	0.0	0.7250
No log	0.8955	60	0.7334	0.0	0.7229
No log	0.9254	62	0.7299	0.0	0.7193
No log	0.9552	64	0.7249	0.0	0.7143
No log	0.9851	66	0.7242	0.0	0.7136

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1