arabert_cross_vocabulary_task7_fold3

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7581
• Qwk: 0.0
• Mse: 0.7581

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0308	2	7.8306	-0.0005	7.8306
No log	0.0615	4	4.1936	0.0	4.1936
No log	0.0923	6	2.0851	0.0593	2.0851
No log	0.1231	8	1.2404	0.0	1.2404
No log	0.1538	10	0.8586	0.0093	0.8586
No log	0.1846	12	0.7508	0.0707	0.7508
No log	0.2154	14	0.7356	0.0372	0.7356
No log	0.2462	16	0.8138	0.1082	0.8138
No log	0.2769	18	0.9262	-0.0080	0.9262
No log	0.3077	20	0.8710	-0.0045	0.8710
No log	0.3385	22	0.7502	-0.0408	0.7502
No log	0.3692	24	0.7389	0.0	0.7389
No log	0.4	26	0.7855	0.0	0.7855
No log	0.4308	28	0.7754	0.0	0.7754
No log	0.4615	30	0.7485	0.0	0.7485
No log	0.4923	32	0.7451	0.0	0.7451
No log	0.5231	34	0.7615	0.0	0.7615
No log	0.5538	36	0.7804	0.0	0.7804
No log	0.5846	38	0.7731	0.0	0.7731
No log	0.6154	40	0.7812	0.0	0.7812
No log	0.6462	42	0.8026	0.0	0.8026
No log	0.6769	44	0.8454	0.0	0.8454
No log	0.7077	46	0.8541	0.0	0.8541
No log	0.7385	48	0.8250	0.0	0.8250
No log	0.7692	50	0.8021	0.0	0.8021
No log	0.8	52	0.7828	0.0	0.7828
No log	0.8308	54	0.7701	0.0	0.7701
No log	0.8615	56	0.7633	0.0	0.7633
No log	0.8923	58	0.7561	0.0	0.7561
No log	0.9231	60	0.7554	0.0	0.7554
No log	0.9538	62	0.7563	0.0	0.7563
No log	0.9846	64	0.7581	0.0	0.7581

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1