metadata

base_model: aubmindlab/bert-base-arabertv02
tags:
  - generated_from_trainer
model-index:
  - name: arabert_cross_vocabulary_task1_fold3
    results: []

arabert_cross_vocabulary_task1_fold3

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.6330
Qwk: 0.8116
Mse: 0.6330

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0351	2	3.3598	0.0190	3.3598
No log	0.0702	4	2.4609	0.1303	2.4609
No log	0.1053	6	1.4030	0.3203	1.4030
No log	0.1404	8	1.5047	0.4056	1.5047
No log	0.1754	10	1.6841	0.5201	1.6841
No log	0.2105	12	1.2050	0.4844	1.2050
No log	0.2456	14	1.0340	0.4610	1.0340
No log	0.2807	16	0.9004	0.5199	0.9004
No log	0.3158	18	0.8944	0.7299	0.8944
No log	0.3509	20	0.7872	0.7550	0.7872
No log	0.3860	22	0.6647	0.7227	0.6647
No log	0.4211	24	0.6390	0.7315	0.6390
No log	0.4561	26	0.6404	0.7837	0.6404
No log	0.4912	28	0.8258	0.8035	0.8258
No log	0.5263	30	0.8849	0.8022	0.8849
No log	0.5614	32	0.7788	0.8146	0.7788
No log	0.5965	34	0.6878	0.7879	0.6878
No log	0.6316	36	0.6812	0.7867	0.6812
No log	0.6667	38	0.7533	0.8236	0.7533
No log	0.7018	40	0.8468	0.8151	0.8468
No log	0.7368	42	0.8501	0.8191	0.8501
No log	0.7719	44	0.8291	0.8174	0.8291
No log	0.8070	46	0.7683	0.8084	0.7683
No log	0.8421	48	0.7140	0.8190	0.7140
No log	0.8772	50	0.6780	0.8204	0.6780
No log	0.9123	52	0.6546	0.8181	0.6546
No log	0.9474	54	0.6399	0.8097	0.6399
No log	0.9825	56	0.6330	0.8116	0.6330

Framework versions

Transformers 4.44.0
Pytorch 2.4.0
Datasets 2.21.0
Tokenizers 0.19.1