Hugging Face's logo

csikasote
/
xls-r-1b-bem-natbed-combined-model

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
wav2vec2
natbed
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
xls-r-1b-bem-natbed-combined-model / README.md
csikasote's picture
csikasote
End of training
01fb589 verified
preview code
|
raw
history blame contribute delete
2.9 kB
metadata
library_name: transformers
license: apache-2.0
base_model: facebook/wav2vec2-xls-r-1b
tags:
  - automatic-speech-recognition
  - natbed
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: xls-r-1b-bem-natbed-combined-model
    results: []

xls-r-1b-bem-natbed-combined-model

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the NATBED - BEM dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7026
  • Wer: 0.7512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 30.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
5.0296 0.2503 100 2.8071 1.0
1.5792 0.5006 200 1.1184 0.9645
1.1422 0.7509 300 1.0389 0.9606
0.9883 1.0013 400 1.0494 0.9989
0.8999 1.2516 500 0.8692 0.8683
0.9135 1.5019 600 0.8564 0.8430
0.8898 1.7522 700 0.8451 0.8522
0.9089 2.0025 800 0.8857 0.8485
0.8292 2.2528 900 0.8662 0.8580
0.7921 2.5031 1000 0.7964 0.7969
0.7983 2.7534 1100 0.7896 0.7951
0.7946 3.0038 1200 0.7667 0.7947
0.7488 3.2541 1300 0.8180 0.8495
0.7428 3.5044 1400 0.7548 0.7688
0.7256 3.7547 1500 0.7258 0.7596
0.741 4.0050 1600 0.7665 0.7718
0.6775 4.2553 1700 0.7922 0.7775
0.6795 4.5056 1800 0.7026 0.7512
0.683 4.7559 1900 0.7051 0.7225
0.6838 5.0063 2000 0.7196 0.7503
0.6005 5.2566 2100 0.7032 0.7424

Framework versions

  • Transformers 4.46.0.dev0
  • Pytorch 2.4.1+cu121
  • Datasets 3.0.1
  • Tokenizers 0.20.0