arabert_cross_vocabulary_task7_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6468
• Qwk: 0.5330
• Mse: 0.6465

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0351	2	7.0706	0.0013	7.0661
No log	0.0702	4	3.7734	0.0327	3.7717
No log	0.1053	6	1.8592	0.0683	1.8601
No log	0.1404	8	1.0155	0.2390	1.0154
No log	0.1754	10	1.1406	0.1653	1.1393
No log	0.2105	12	1.3051	0.1603	1.3037
No log	0.2456	14	1.6813	0.1353	1.6795
No log	0.2807	16	1.9830	0.1290	1.9812
No log	0.3158	18	1.4915	0.2062	1.4903
No log	0.3509	20	0.8770	0.3141	0.8766
No log	0.3860	22	0.7757	0.3596	0.7754
No log	0.4211	24	0.7897	0.3469	0.7894
No log	0.4561	26	0.7860	0.3869	0.7857
No log	0.4912	28	1.0219	0.3741	1.0213
No log	0.5263	30	1.4552	0.3028	1.4543
No log	0.5614	32	1.7633	0.2481	1.7624
No log	0.5965	34	1.7356	0.2503	1.7348
No log	0.6316	36	1.3222	0.3331	1.3216
No log	0.6667	38	0.9190	0.3911	0.9185
No log	0.7018	40	0.7186	0.4702	0.7182
No log	0.7368	42	0.6343	0.5308	0.6340
No log	0.7719	44	0.6126	0.5621	0.6123
No log	0.8070	46	0.6041	0.5797	0.6039
No log	0.8421	48	0.6057	0.5797	0.6055
No log	0.8772	50	0.6117	0.5643	0.6115
No log	0.9123	52	0.6235	0.5650	0.6232
No log	0.9474	54	0.6369	0.5445	0.6367
No log	0.9825	56	0.6468	0.5330	0.6465

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1