multimodal-traj-class-no-numtransform

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.1964
  • Acc: 0.7237
  • Relacc: 0.8446
  • Num Fours: 617
  • Mcc: 0.6029

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss Acc Relacc Num Fours Mcc
0.9461 1.0 1212 0.8645 0.6752 0.7586 680 0.5115
0.8387 2.0 2424 0.7880 0.6979 0.7889 630 0.5526
0.7565 3.0 3636 0.7489 0.7183 0.8015 636 0.5851
0.6997 4.0 4848 0.7542 0.7061 0.7908 574 0.5569
0.6516 5.0 6060 0.6806 0.7388 0.8192 660 0.6176
0.6049 6.0 7272 0.6898 0.7406 0.8395 638 0.6269
0.5526 7.0 8484 0.6848 0.7408 0.8413 648 0.6288
0.5343 8.0 9696 0.6904 0.7359 0.8413 645 0.6207
0.4855 9.0 10908 0.7219 0.7400 0.8456 587 0.6253
0.4618 10.0 12120 0.7310 0.7464 0.8448 624 0.6314
0.4326 11.0 13332 0.7298 0.7575 0.8508 658 0.6536
0.4098 12.0 14544 0.8706 0.7266 0.8395 611 0.6026
0.3707 13.0 15756 0.8682 0.7431 0.8415 629 0.6260
0.3377 14.0 16968 0.9299 0.7371 0.8467 590 0.6220
0.315 15.0 18180 0.9393 0.7365 0.8463 635 0.6190
0.2984 16.0 19392 1.0106 0.7348 0.8426 593 0.6134
0.2804 17.0 20604 1.0719 0.7307 0.8465 623 0.6118
0.2644 18.0 21816 1.1245 0.7280 0.8446 642 0.6117
0.2469 19.0 23028 1.1745 0.7258 0.8430 619 0.6044
0.2273 20.0 24240 1.1964 0.7237 0.8446 617 0.6029

Framework versions

  • Transformers 4.25.1
  • Pytorch 1.13.0+cu116
  • Datasets 2.7.1
  • Tokenizers 0.13.2
Downloads last month
11
Inference API
Unable to determine this model’s pipeline type. Check the docs .