arabert_cross_vocabulary_task5_fold3

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8004
• Qwk: 0.0
• Mse: 0.8004

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0308	2	8.2531	0.0	8.2531
No log	0.0615	4	4.7718	0.0139	4.7718
No log	0.0923	6	2.3636	0.0073	2.3636
No log	0.1231	8	1.1904	0.0	1.1904
No log	0.1538	10	0.7960	0.0079	0.7960
No log	0.1846	12	0.7511	0.0372	0.7511
No log	0.2154	14	0.7663	0.0434	0.7663
No log	0.2462	16	0.7970	0.0	0.7970
No log	0.2769	18	0.8420	0.0	0.8420
No log	0.3077	20	0.8106	0.0	0.8106
No log	0.3385	22	0.7687	0.0	0.7687
No log	0.3692	24	0.7576	0.0	0.7576
No log	0.4	26	0.7671	-0.0229	0.7671
No log	0.4308	28	0.7694	0.0	0.7694
No log	0.4615	30	0.7718	0.0	0.7718
No log	0.4923	32	0.7692	0.0	0.7692
No log	0.5231	34	0.7614	0.0	0.7614
No log	0.5538	36	0.7606	0.0	0.7606
No log	0.5846	38	0.7654	0.0	0.7654
No log	0.6154	40	0.7697	0.0	0.7697
No log	0.6462	42	0.7654	0.0	0.7654
No log	0.6769	44	0.7659	0.0	0.7659
No log	0.7077	46	0.7694	0.0	0.7694
No log	0.7385	48	0.7700	0.0	0.7700
No log	0.7692	50	0.7602	0.0	0.7602
No log	0.8	52	0.7497	0.0071	0.7497
No log	0.8308	54	0.7542	0.0071	0.7542
No log	0.8615	56	0.7680	0.0	0.7680
No log	0.8923	58	0.7806	0.0	0.7806
No log	0.9231	60	0.7911	0.0	0.7911
No log	0.9538	62	0.7981	0.0	0.7981
No log	0.9846	64	0.8004	0.0	0.8004

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1