Edit model card

arabert_baseline_vocabulary_task5_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5505
  • Qwk: 0.6767
  • Mse: 0.5505

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Qwk Mse
No log 0.3333 2 3.3545 0.0255 3.3545
No log 0.6667 4 1.6087 0.0 1.6087
No log 1.0 6 0.8536 0.2689 0.8536
No log 1.3333 8 0.7544 0.2289 0.7544
No log 1.6667 10 0.6504 0.2689 0.6504
No log 2.0 12 0.5598 0.3623 0.5598
No log 2.3333 14 0.5291 0.3623 0.5291
No log 2.6667 16 0.6356 0.4179 0.6356
No log 3.0 18 0.8795 0.2857 0.8795
No log 3.3333 20 0.7838 0.4434 0.7838
No log 3.6667 22 0.5344 0.6809 0.5344
No log 4.0 24 0.5107 0.6822 0.5107
No log 4.3333 26 0.5455 0.6822 0.5455
No log 4.6667 28 0.5959 0.6822 0.5959
No log 5.0 30 0.6155 0.6091 0.6155
No log 5.3333 32 0.5993 0.6146 0.5993
No log 5.6667 34 0.6037 0.7028 0.6037
No log 6.0 36 0.6613 0.6767 0.6613
No log 6.3333 38 0.6632 0.7005 0.6632
No log 6.6667 40 0.6185 0.6822 0.6185
No log 7.0 42 0.5683 0.7028 0.5683
No log 7.3333 44 0.5821 0.6040 0.5821
No log 7.6667 46 0.5844 0.6040 0.5844
No log 8.0 48 0.5500 0.6844 0.5500
No log 8.3333 50 0.5326 0.7028 0.5326
No log 8.6667 52 0.5428 0.6767 0.5428
No log 9.0 54 0.5510 0.6767 0.5510
No log 9.3333 56 0.5525 0.6767 0.5525
No log 9.6667 58 0.5510 0.6767 0.5510
No log 10.0 60 0.5505 0.6767 0.5505

Framework versions

  • Transformers 4.44.0
  • Pytorch 2.4.0
  • Datasets 2.21.0
  • Tokenizers 0.19.1
Downloads last month
5
Safetensors
Model size
135M params
Tensor type
F32
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for salbatarni/arabert_baseline_vocabulary_task5_fold1

Finetuned
(949)
this model