arabert_cross_vocabulary_task5_fold2

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6937
• Qwk: 0.0
• Mse: 0.6827

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0317	2	4.6009	0.0009	4.6270
No log	0.0635	4	1.4046	-0.0070	1.4154
No log	0.0952	6	0.7366	-0.0031	0.7371
No log	0.1270	8	0.6366	-0.0370	0.6297
No log	0.1587	10	0.7085	-0.0259	0.6993
No log	0.1905	12	0.7820	-0.0140	0.7757
No log	0.2222	14	0.7393	-0.0458	0.7348
No log	0.2540	16	0.5809	0.0613	0.5748
No log	0.2857	18	0.5907	0.0	0.5826
No log	0.3175	20	0.5968	0.0	0.5879
No log	0.3492	22	0.5997	0.0	0.5907
No log	0.3810	24	0.6179	0.0	0.6081
No log	0.4127	26	0.6180	0.0	0.6082
No log	0.4444	28	0.6050	0.0	0.5957
No log	0.4762	30	0.6052	0.0	0.5959
No log	0.5079	32	0.5967	0.0	0.5878
No log	0.5397	34	0.6247	0.0	0.6152
No log	0.5714	36	0.6697	0.0	0.6594
No log	0.6032	38	0.7455	0.0	0.7341
No log	0.6349	40	0.7874	0.0	0.7757
No log	0.6667	42	0.7649	0.0	0.7537
No log	0.6984	44	0.7194	0.0	0.7089
No log	0.7302	46	0.6683	0.0	0.6584
No log	0.7619	48	0.6472	0.0	0.6374
No log	0.7937	50	0.6425	0.0	0.6324
No log	0.8254	52	0.6567	0.0	0.6463
No log	0.8571	54	0.6712	0.0	0.6605
No log	0.8889	56	0.6746	0.0	0.6639
No log	0.9206	58	0.6773	0.0	0.6666
No log	0.9524	60	0.6866	0.0	0.6758
No log	0.9841	62	0.6937	0.0	0.6827

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1