arabert_cross_vocabulary_task3_fold5

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5424
• Qwk: 0.8451
• Mse: 0.5424

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0317	2	2.5435	0.0266	2.5435
No log	0.0635	4	1.8405	0.0779	1.8405
No log	0.0952	6	1.1417	0.3446	1.1417
No log	0.1270	8	1.3024	0.3139	1.3024
No log	0.1587	10	1.4661	0.4195	1.4661
No log	0.1905	12	1.4593	0.5454	1.4593
No log	0.2222	14	1.0707	0.6205	1.0707
No log	0.2540	16	0.8275	0.7241	0.8275
No log	0.2857	18	0.7219	0.7795	0.7219
No log	0.3175	20	0.6521	0.7938	0.6521
No log	0.3492	22	0.6278	0.8032	0.6278
No log	0.3810	24	0.6249	0.8114	0.6249
No log	0.4127	26	0.6081	0.8169	0.6081
No log	0.4444	28	0.6712	0.8158	0.6712
No log	0.4762	30	0.6982	0.8160	0.6982
No log	0.5079	32	0.5559	0.8283	0.5559
No log	0.5397	34	0.4227	0.7642	0.4227
No log	0.5714	36	0.4112	0.7538	0.4112
No log	0.6032	38	0.3988	0.7554	0.3988
No log	0.6349	40	0.3927	0.7690	0.3927
No log	0.6667	42	0.3945	0.7457	0.3945
No log	0.6984	44	0.4001	0.7010	0.4001
No log	0.7302	46	0.4142	0.6805	0.4142
No log	0.7619	48	0.4142	0.6989	0.4142
No log	0.7937	50	0.4157	0.7460	0.4157
No log	0.8254	52	0.4406	0.7973	0.4406
No log	0.8571	54	0.4698	0.7995	0.4698
No log	0.8889	56	0.5044	0.8216	0.5044
No log	0.9206	58	0.5361	0.8422	0.5361
No log	0.9524	60	0.5468	0.8468	0.5468
No log	0.9841	62	0.5424	0.8451	0.5424

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1