Hugging Face's logo

HiTZ
/
whisper-large-eu

PyTorch
Basque
whisper
whisper-event
Generated from Trainer
Eval Results
Model card Files
xet
Community
1
whisper-large-eu / README.md
asierhv's picture
asierhv
added funding
e7066b4 verified
preview code
|
raw
history blame contribute delete
7.24 kB
metadata
language:
  - eu
license: apache-2.0
base_model: openai/whisper-large
tags:
  - whisper-event
  - generated_from_trainer
datasets:
  - mozilla-foundation/common_voice_13_0
metrics:
  - wer
model-index:
  - name: Whisper Large Basque
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: mozilla-foundation/common_voice_13_0 eu
          type: mozilla-foundation/common_voice_13_0
          config: eu
          split: test
          args: eu
        metrics:
          - name: Wer
            type: wer
            value: 12.234193365466401

Whisper Large Basque

Model summary

Whisper Large Basque is an automatic speech recognition (ASR) model for Basque (eu) speech. It is fine-tuned from [openai/whisper-large] on the Basque portion of Mozilla Common Voice 13.0, achieving a Word Error Rate (WER) of 12.23% on the Common Voice evaluation split.

This model provides high-quality transcription for Basque speech, offering substantial improvements in accuracy over smaller Whisper variants while suitable for offline and batch processing tasks.

Model description

  • Architecture: Transformer-based encoder–decoder (Whisper)
  • Base model: openai/whisper-large
  • Language: Basque (eu)
  • Task: Automatic Speech Recognition (ASR)
  • Output: Text transcription in Basque
  • Decoding: Autoregressive sequence-to-sequence decoding

Leveraging Whisper’s multilingual pretraining, this large model is fine-tuned on Basque speech data to deliver highly accurate transcription for a low-resource language, suitable for research, media, and archival use cases.

Intended use

Primary use cases

  • High-quality transcription of Basque audio recordings
  • Offline or batch ASR pipelines
  • Research and development in Basque ASR
  • Media, educational, and archival transcription tasks

Intended users

  • Researchers working on Basque or low-resource ASR
  • Developers building Basque speech applications
  • Academic and institutional users

Out-of-scope use

  • Real-time or low-latency ASR without optimization
  • Speech translation tasks
  • Safety-critical applications without validation

Limitations and known issues

  • Performance may degrade on:
    • Noisy or low-quality recordings
    • Conversational or spontaneous speech
    • Accents underrepresented in Common Voice
  • While highly accurate, transcription errors may still occur under challenging acoustic conditions
  • Dataset biases from Common Voice may be reflected in outputs

Users are encouraged to evaluate the model on their own data before deployment.

Training and evaluation data

Training data

  • Dataset: Mozilla Common Voice 13.0 (Basque subset)
  • Data type: Crowd-sourced, read speech
  • Preprocessing:
    • Audio resampled to 16 kHz
    • Text normalized using Whisper tokenizer
    • Filtering of invalid or problematic samples

Evaluation data

  • Dataset: Mozilla Common Voice 13.0 (Basque evaluation split)
  • Metric: Word Error Rate (WER)

Evaluation results

Metric Value
WER (eval) 12.23%

These results indicate state-of-the-art performance for Basque ASR using a large Whisper model.

Training procedure

Training hyperparameters

  • Learning rate: 1e-5
  • Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
  • LR scheduler: Linear
  • Warmup steps: 500
  • Training steps: 20,000
  • Train batch size: 32
  • Gradient accumulation steps: 2
  • Total effective batch size: 64
  • Evaluation batch size: 16
  • Seed: 42

Training results (summary)

Training Loss Epoch Step Validation Loss WER
0.0196 4.01 1000 0.2825 15.4725
0.0039 9.01 2000 0.3072 14.2270
0.0031 14.01 3000 0.3170 13.7652
0.0023 19.0 4000 0.3310 13.6640
0.0014 24.0 5000 0.3384 13.5749
0.0034 29.0 6000 0.3425 13.7450
0.0011 33.01 7000 0.3476 13.0990
0.001 38.01 8000 0.3432 13.0990
0.0004 43.01 9000 0.3524 12.8033
0.0017 48.01 10000 0.3620 13.3946
0.0003 53.0 11000 0.3564 12.6190
0.0001 58.0 12000 0.3675 12.6352
0.0 63.0 13000 0.3878 12.4286
0.0 67.01 14000 0.3996 12.3577
0.0 72.01 15000 0.4088 12.3456
0.0 77.01 16000 0.4167 12.3091
0.0 82.01 17000 0.4241 12.3112
0.0 87.0 18000 0.4302 12.3193
0.0 92.0 19000 0.4351 12.2565
0.0 97.0 20000 0.4369 12.2342

Framework versions

  • Transformers 4.33.0.dev0
  • PyTorch 2.0.1+cu117
  • Datasets 2.14.4
  • Tokenizers 0.13.3

How to use 

from transformers import pipeline

hf_model = "HiTZ/whisper-large-eu"  # replace with actual repo ID
device = 0  # set to -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device
)

result = pipe("audio.wav")
print(result["text"])

Ethical considerations and risks

  • This model transcribes speech and may process personal data.
  • Users should ensure compliance with applicable data protection laws (e.g., GDPR).
  • The model should not be used for surveillance or non-consensual audio processing.

Citation

If you use this model in your research, please cite:

@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Please, check the related paper preprint in arXiv:2503.23542 for more details.

License

This model is available under the Apache-2.0 License. You are free to use, modify, and distribute this model as long as you credit the original creators.

Contact and attribution

  • Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
  • Base model: OpenAI Whisper
  • Dataset: Mozilla Common Voice

For questions or issues, please open an issue in the model repository.

Funding

This project with reference 2022/TL22/00215335 has been parcially funded by the Ministerio de Transformación Digital and by the Plan de Recuperación, Transformación y Resiliencia – Funded by the European Union – NextGenerationEU ILENIA and by the project IkerGaitu funded by the Basque Government. This model was trained at Hyperion, one of the high-performance computing (HPC) systems hosted by the DIPC Supercomputing Center.