arabert_cross_vocabulary_task4_fold2

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4786
• Qwk: 0.0414
• Mse: 1.4653

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0282	2	7.1347	0.0039	7.1562
No log	0.0563	4	3.4551	0.0095	3.4688
No log	0.0845	6	1.4618	0.0540	1.4670
No log	0.1127	8	0.7258	-0.0161	0.7252
No log	0.1408	10	0.7701	-0.0572	0.7709
No log	0.1690	12	1.0250	-0.0732	1.0315
No log	0.1972	14	0.7175	-0.0348	0.7181
No log	0.2254	16	0.6112	0.0365	0.6099
No log	0.2535	18	0.5589	0.0853	0.5572
No log	0.2817	20	0.5273	0.0940	0.5251
No log	0.3099	22	0.5480	0.0098	0.5432
No log	0.3380	24	0.7106	0.0098	0.7034
No log	0.3662	26	0.9344	0.0202	0.9253
No log	0.3944	28	1.1676	0.1456	1.1562
No log	0.4225	30	1.2628	0.1249	1.2507
No log	0.4507	32	1.0404	0.1021	1.0296
No log	0.4789	34	1.0395	0.1112	1.0292
No log	0.5070	36	1.1961	0.1187	1.1868
No log	0.5352	38	1.1022	0.1439	1.0951
No log	0.5634	40	0.8840	-0.0190	0.8791
No log	0.5915	42	0.7380	-0.0094	0.7349
No log	0.6197	44	0.7231	-0.0336	0.7198
No log	0.6479	46	0.7578	-0.0424	0.7530
No log	0.6761	48	0.9652	-0.0262	0.9566
No log	0.7042	50	1.1114	-0.1252	1.1008
No log	0.7324	52	1.2059	0.0116	1.1948
No log	0.7606	54	1.3383	-0.0009	1.3265
No log	0.7887	56	1.4278	0.0369	1.4156
No log	0.8169	58	1.4154	0.0855	1.4032
No log	0.8451	60	1.3691	0.1138	1.3571
No log	0.8732	62	1.3963	0.0789	1.3841
No log	0.9014	64	1.4466	0.0640	1.4340
No log	0.9296	66	1.4766	0.0358	1.4637
No log	0.9577	68	1.4844	0.0203	1.4712
No log	0.9859	70	1.4786	0.0414	1.4653

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1