arabert_cross_vocabulary_task1_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8237
• Qwk: 0.0005
• Mse: 0.8231

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.125	2	6.3728	-0.0031	6.3683
No log	0.25	4	2.4100	-0.0135	2.4074
No log	0.375	6	0.8300	0.0029	0.8281
No log	0.5	8	0.5694	0.0422	0.5684
No log	0.625	10	0.5771	0.0819	0.5758
No log	0.75	12	0.5787	0.0393	0.5780
No log	0.875	14	0.5560	0.0075	0.5555
No log	1.0	16	0.5627	-0.0083	0.5623
No log	1.125	18	0.6677	0.0	0.6674
No log	1.25	20	0.7346	0.0	0.7343
No log	1.375	22	0.6544	0.0	0.6540
No log	1.5	24	0.7414	0.0	0.7411
No log	1.625	26	0.7091	0.0	0.7088
No log	1.75	28	0.5841	0.0	0.5836
No log	1.875	30	0.5323	0.0422	0.5314
No log	2.0	32	0.5359	0.0488	0.5349
No log	2.125	34	0.5392	0.0	0.5385
No log	2.25	36	0.6346	0.0	0.6342
No log	2.375	38	0.7730	0.0	0.7727
No log	2.5	40	0.7599	0.0	0.7595
No log	2.625	42	0.6719	0.0	0.6714
No log	2.75	44	0.7595	0.0	0.7591
No log	2.875	46	0.8678	0.0202	0.8675
No log	3.0	48	0.8677	0.0151	0.8674
No log	3.125	50	0.9100	0.0193	0.9096
No log	3.25	52	1.2346	0.0466	1.2345
No log	3.375	54	1.3974	0.1074	1.3974
No log	3.5	56	1.1473	0.0657	1.1472
No log	3.625	58	0.7783	-0.0050	0.7779
No log	3.75	60	0.6393	0.0	0.6387
No log	3.875	62	0.6653	0.0	0.6648
No log	4.0	64	0.6339	0.0	0.6334
No log	4.125	66	0.6583	0.0	0.6578
No log	4.25	68	0.7357	0.0	0.7353
No log	4.375	70	0.9251	-0.0269	0.9248
No log	4.5	72	1.0004	0.0702	1.0001
No log	4.625	74	1.0757	0.0049	1.0754
No log	4.75	76	1.0881	0.0049	1.0878
No log	4.875	78	0.8102	-0.0195	0.8097
No log	5.0	80	0.6847	0.0	0.6840
No log	5.125	82	0.7337	-0.0050	0.7331
No log	5.25	84	0.7008	0.0	0.7001
No log	5.375	86	0.7016	-0.0050	0.7009
No log	5.5	88	0.7481	-0.0195	0.7475
No log	5.625	90	0.7384	0.0051	0.7377
No log	5.75	92	0.7519	0.0003	0.7511
No log	5.875	94	0.9434	0.0457	0.9429
No log	6.0	96	1.1388	0.1084	1.1387
No log	6.125	98	1.1119	0.1087	1.1118
No log	6.25	100	0.9851	0.0611	0.9849
No log	6.375	102	0.8431	0.0327	0.8426
No log	6.5	104	0.7529	-0.0242	0.7523
No log	6.625	106	0.7658	-0.0242	0.7653
No log	6.75	108	0.8717	0.0376	0.8713
No log	6.875	110	0.9321	0.0376	0.9319
No log	7.0	112	0.9103	0.0426	0.9101
No log	7.125	114	0.8699	0.0193	0.8696
No log	7.25	116	0.7446	-0.0195	0.7440
No log	7.375	118	0.6938	-0.0446	0.6931
No log	7.5	120	0.6932	-0.0446	0.6926
No log	7.625	122	0.7665	-0.0242	0.7659
No log	7.75	124	0.9012	-0.0078	0.9008
No log	7.875	126	0.9982	0.0480	0.9978
No log	8.0	128	1.0450	0.0602	1.0446
No log	8.125	130	1.0164	0.0643	1.0160
No log	8.25	132	0.9926	0.0728	0.9921
No log	8.375	134	0.9480	0.0328	0.9475
No log	8.5	136	0.9640	0.0245	0.9635
No log	8.625	138	0.9946	0.0501	0.9942
No log	8.75	140	0.9731	0.0245	0.9727
No log	8.875	142	0.9147	0.0213	0.9142
No log	9.0	144	0.8829	0.0177	0.8824
No log	9.125	146	0.8432	0.0231	0.8426
No log	9.25	148	0.8012	0.0050	0.8006
No log	9.375	150	0.7936	-0.0182	0.7929
No log	9.5	152	0.7870	-0.0182	0.7863
No log	9.625	154	0.8001	0.0050	0.7994
No log	9.75	156	0.8093	0.0050	0.8086
No log	9.875	158	0.8204	0.0005	0.8198
No log	10.0	160	0.8237	0.0005	0.8231

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1