arabert_cross_vocabulary_task5_fold4

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5724
• Qwk: 0.8274
• Mse: 0.5724

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0323	2	4.0786	-0.0202	4.0786
No log	0.0645	4	2.3283	0.0109	2.3283
No log	0.0968	6	1.2785	0.2841	1.2785
No log	0.1290	8	0.8433	0.4083	0.8433
No log	0.1613	10	1.0059	0.5899	1.0059
No log	0.1935	12	1.1298	0.6125	1.1298
No log	0.2258	14	0.8824	0.7014	0.8824
No log	0.2581	16	0.9194	0.7028	0.9194
No log	0.2903	18	0.9116	0.7100	0.9116
No log	0.3226	20	0.6668	0.7739	0.6668
No log	0.3548	22	0.4256	0.7812	0.4256
No log	0.3871	24	0.4050	0.7574	0.4050
No log	0.4194	26	0.5814	0.7955	0.5814
No log	0.4516	28	1.0279	0.7514	1.0279
No log	0.4839	30	1.0452	0.7607	1.0452
No log	0.5161	32	0.7165	0.8089	0.7165
No log	0.5484	34	0.4458	0.7951	0.4458
No log	0.5806	36	0.3735	0.7660	0.3735
No log	0.6129	38	0.3982	0.7985	0.3982
No log	0.6452	40	0.5173	0.8081	0.5173
No log	0.6774	42	0.7723	0.7985	0.7723
No log	0.7097	44	0.9254	0.7695	0.9254
No log	0.7419	46	1.0677	0.7448	1.0677
No log	0.7742	48	1.0524	0.7448	1.0524
No log	0.8065	50	0.9408	0.7762	0.9408
No log	0.8387	52	0.8100	0.7901	0.8100
No log	0.8710	54	0.7061	0.8033	0.7061
No log	0.9032	56	0.6220	0.8252	0.6220
No log	0.9355	58	0.5830	0.8299	0.5830
No log	0.9677	60	0.5721	0.8239	0.5721
No log	1.0	62	0.5724	0.8274	0.5724

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1