arabert_cross_vocabulary_task2_fold3

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7212
• Qwk: 0.7957
• Mse: 0.7212

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0351	2	3.3323	0.0213	3.3323
No log	0.0702	4	2.0333	0.1468	2.0333
No log	0.1053	6	1.4923	0.3102	1.4923
No log	0.1404	8	1.6734	0.3305	1.6734
No log	0.1754	10	1.7352	0.2966	1.7352
No log	0.2105	12	1.6360	0.2588	1.6360
No log	0.2456	14	1.5076	0.2925	1.5076
No log	0.2807	16	1.3433	0.3646	1.3433
No log	0.3158	18	1.2343	0.4245	1.2343
No log	0.3509	20	1.1517	0.5033	1.1517
No log	0.3860	22	1.1391	0.6144	1.1391
No log	0.4211	24	1.1438	0.6514	1.1438
No log	0.4561	26	1.1808	0.6567	1.1808
No log	0.4912	28	1.1197	0.7055	1.1197
No log	0.5263	30	0.9660	0.7854	0.9660
No log	0.5614	32	0.8893	0.7754	0.8893
No log	0.5965	34	0.8546	0.7866	0.8546
No log	0.6316	36	0.9013	0.7896	0.9013
No log	0.6667	38	0.9216	0.8034	0.9216
No log	0.7018	40	0.9057	0.7971	0.9057
No log	0.7368	42	0.8462	0.7814	0.8462
No log	0.7719	44	0.7936	0.7892	0.7936
No log	0.8070	46	0.7517	0.8069	0.7517
No log	0.8421	48	0.7133	0.7975	0.7133
No log	0.8772	50	0.6939	0.7967	0.6939
No log	0.9123	52	0.7043	0.7937	0.7043
No log	0.9474	54	0.7171	0.7957	0.7171
No log	0.9825	56	0.7212	0.7957	0.7212

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1