gossminn's picture
update model card README.md
336a000
metadata
license: mit
tags:
  - generated_from_trainer
model-index:
  - name: predict-perception-bertino-focus-assassin
    results: []

predict-perception-bertino-focus-assassin

This model is a fine-tuned version of indigo-ai/BERTino on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3409
  • R2: 0.3205

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 20
  • eval_batch_size: 8
  • seed: 1996
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 47

Training results

Training Loss Epoch Step Validation Loss R2
0.5573 1.0 14 0.4856 0.0321
0.1739 2.0 28 0.4735 0.0562
0.0813 3.0 42 0.3416 0.3191
0.0764 4.0 56 0.3613 0.2799
0.0516 5.0 70 0.3264 0.3495
0.0459 6.0 84 0.4193 0.1643
0.0414 7.0 98 0.3502 0.3019
0.028 8.0 112 0.3361 0.3301
0.0281 9.0 126 0.3610 0.2804
0.027 10.0 140 0.3523 0.2978
0.0216 11.0 154 0.3440 0.3143
0.0181 12.0 168 0.3506 0.3012
0.013 13.0 182 0.3299 0.3424
0.0116 14.0 196 0.3611 0.2803
0.0118 15.0 210 0.3505 0.3013
0.0139 16.0 224 0.3529 0.2967
0.0099 17.0 238 0.3536 0.2952
0.0096 18.0 252 0.3542 0.2941
0.0107 19.0 266 0.3770 0.2486
0.0088 20.0 280 0.3467 0.3091
0.0065 21.0 294 0.3327 0.3369
0.0073 22.0 308 0.3479 0.3066
0.0062 23.0 322 0.3566 0.2893
0.0063 24.0 336 0.3503 0.3019
0.0057 25.0 350 0.3371 0.3282
0.0049 26.0 364 0.3334 0.3355
0.0045 27.0 378 0.3399 0.3225
0.0049 28.0 392 0.3379 0.3266
0.0049 29.0 406 0.3377 0.3268
0.0055 30.0 420 0.3357 0.3309
0.005 31.0 434 0.3394 0.3235
0.0046 32.0 448 0.3432 0.3159
0.0048 33.0 462 0.3427 0.3169
0.0041 34.0 476 0.3450 0.3123
0.0041 35.0 490 0.3436 0.3151
0.0051 36.0 504 0.3394 0.3234
0.0037 37.0 518 0.3370 0.3283
0.004 38.0 532 0.3370 0.3284
0.0033 39.0 546 0.3339 0.3344
0.0034 40.0 560 0.3335 0.3352
0.003 41.0 574 0.3373 0.3276
0.0035 42.0 588 0.3380 0.3264
0.0032 43.0 602 0.3382 0.3259
0.0034 44.0 616 0.3432 0.3158
0.003 45.0 630 0.3421 0.3181
0.0027 46.0 644 0.3410 0.3203
0.0037 47.0 658 0.3409 0.3205

Framework versions

  • Transformers 4.16.2
  • Pytorch 1.10.2+cu113
  • Datasets 1.18.3
  • Tokenizers 0.11.0