Beijuka's picture
End of training
92bf60c verified
metadata
library_name: transformers
language:
  - af
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
  - generated_from_trainer
datasets:
  - NCHLT_speech_corpus
metrics:
  - wer
model-index:
  - name: facebook mms-1b-all Afrikaans - Beijuka Bruno
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: NCHLT_speech_corpus/Afrikaans
          type: NCHLT_speech_corpus
        metrics:
          - name: Wer
            type: wer
            value: 0.7866727019948915

facebook mms-1b-all Afrikaans - Beijuka Bruno

This model is a fine-tuned version of facebook/mms-1b-all on the NCHLT_speech_corpus/Afrikaans dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8746
  • Wer: 0.7867
  • Cer: 0.1929

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 16
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Wer Cer
0.5068 1.0 3678 0.0972 0.1899 0.0251
0.1819 2.0 7356 0.0805 0.1433 0.0203
0.1591 3.0 11034 0.0739 0.1270 0.0181
0.1449 4.0 14712 0.0700 0.1249 0.0174
0.1355 5.0 18390 0.0673 0.1135 0.0162
0.1283 6.0 22068 0.0655 0.1094 0.0153
0.1218 7.0 25746 0.0605 0.0953 0.0142
0.1171 8.0 29424 0.0580 0.0942 0.0141
0.113 9.0 33102 0.0576 0.0916 0.0132
0.1071 10.0 36780 0.0605 0.1099 0.0143
0.106 11.0 40458 0.0558 0.0878 0.0129
0.1021 12.0 44136 0.0542 0.0810 0.0122
0.0992 13.0 47814 0.0529 0.0807 0.0120
0.0965 14.0 51492 0.0542 0.0884 0.0124
0.0946 15.0 55170 0.0511 0.0798 0.0118
0.0914 16.0 58848 0.0500 0.0756 0.0112
0.0898 17.0 62526 0.0499 0.0774 0.0114
0.0868 18.0 66204 0.0484 0.0706 0.0108
0.085 19.0 69882 0.0495 0.0752 0.0109
0.0843 20.0 73560 0.0492 0.0704 0.0105
0.0822 21.0 77238 0.0480 0.0671 0.0102
0.0807 22.0 80916 0.0473 0.0728 0.0104
0.0794 23.0 84594 0.0462 0.0694 0.0102
0.0781 24.0 88272 0.0460 0.0668 0.0101
0.0771 25.0 91950 0.0463 0.0677 0.0100
0.0754 26.0 95628 0.0439 0.0598 0.0091
0.0745 27.0 99306 0.0463 0.0632 0.0097
0.0738 28.0 102984 0.0455 0.0635 0.0098
0.0718 29.0 106662 0.0455 0.0631 0.0095
0.0707 30.0 110340 0.0433 0.0591 0.0089
0.0696 31.0 114018 0.0454 0.0598 0.0095
0.0685 32.0 117696 0.0446 0.0613 0.0093
0.0674 33.0 121374 0.0437 0.0565 0.0088
0.0682 34.0 125052 0.0467 0.0679 0.0097
0.0648 35.0 128730 0.0467 0.0587 0.0090
0.0658 36.0 132408 0.0439 0.0535 0.0085
0.0633 37.0 136086 0.0448 0.0539 0.0085
0.0639 38.0 139764 0.0440 0.0569 0.0089
0.0619 39.0 143442 0.0458 0.0584 0.0088
0.0622 40.0 147120 0.0449 0.0587 0.0089
0.0618 41.0 150798 0.0461 0.0567 0.0087
0.0614 42.0 154476 0.0461 0.0550 0.0088
0.0611 43.0 158154 0.0427 0.0554 0.0085
0.0588 44.0 161832 0.0436 0.0529 0.0084
0.058 45.0 165510 0.0446 0.0546 0.0085
0.059 46.0 169188 0.0432 0.0543 0.0083

Framework versions

  • Transformers 4.47.0
  • Pytorch 2.1.0+cu118
  • Datasets 3.2.0
  • Tokenizers 0.21.0