whisper-large-ca / README.md

asierhv

added description and "how to use" example

048b950 verified about 8 hours ago

preview code

raw

history blame contribute delete

5.91 kB

metadata

language:
  - ca
license: apache-2.0
base_model: openai/whisper-large
tags:
  - whisper-event
  - generated_from_trainer
datasets:
  - mozilla-foundation/common_voice_13_0
metrics:
  - wer
model-index:
  - name: Whisper Large Catalan
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: mozilla-foundation/common_voice_13_0 ca
          type: mozilla-foundation/common_voice_13_0
          config: ca
          split: test
          args: ca
        metrics:
          - name: Wer
            type: wer
            value: 5.070020005715919

Whisper Large Catalan

Model summary

Whisper Large Catalan is an automatic speech recognition (ASR) model for Catalan (ca) speech. It is fine-tuned from [openai/whisper-large] on the Catalan subset of Mozilla Common Voice 13.0, achieving a Word Error Rate (WER) of 5.070% on the evaluation split.

This model is suitable for high-accuracy transcription and supports longer audio sequences with larger model capacity compared to the medium variant.

Model description

Architecture: Transformer-based encoder–decoder (Whisper)
Base model: openai/whisper-large
Language: Catalan (ca)
Task: Automatic Speech Recognition (ASR)
Output: Text transcription in Catalan
Decoding: Autoregressive sequence-to-sequence decoding

Fine-tuned to improve transcription quality on Catalan audio.

Intended use

Primary use cases

High-accuracy transcription of Catalan audio
Research and development in Catalan ASR
Media, educational, or accessibility applications

Out-of-scope use

Real-time transcription without optimization
Speech translation
Safety-critical applications without further validation

Limitations and known issues

Performance may degrade on:
- Noisy or low-quality recordings
- Conversational or spontaneous speech
- Regional dialects not well represented in Common Voice
Occasional transcription errors on difficult audio

Training and evaluation data

Dataset: Mozilla Common Voice 13.0 (Catalan subset)
Data type: Crowd-sourced, read speech
Preprocessing:
- Audio resampled to 16 kHz
- Text normalized using Whisper tokenizer
- Filtering of invalid or problematic samples
Evaluation metric: Word Error Rate (WER) on held-out evaluation set

Evaluation results

Metric	Value
WER (eval)	5.070%

Training procedure

Training hyperparameters

Learning rate: 1e-5
Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
LR scheduler: Linear
Warmup steps: 500
Training steps: 20,000
Train batch size: 32
Eval batch size: 16
Gradient accumulation steps: 2
Seed: 42

Training results (summary)

Training Loss	Epoch	Step	Validation Loss	WER
0.1059	1.02	1000	0.1744	7.6342
0.0159	3.02	2000	0.1943	7.3850
0.0526	5.02	3000	0.1899	6.8522
0.058	7.02	4000	0.1782	6.7802
0.0161	9.02	5000	0.1995	6.6339
0.065	11.02	6000	0.1563	6.4544
0.082	13.02	7000	0.1789	6.0309
0.0339	15.02	8000	0.1509	5.7554
0.0581	17.01	9000	0.1573	6.0446
0.0181	19.01	10000	0.1838	5.5913
0.0188	21.01	11000	0.1610	5.4804
0.0134	23.01	12000	0.1821	5.3953
0.008	25.01	13000	0.1748	5.3804
0.0071	27.01	14000	0.1858	5.4701
0.0371	29.01	15000	0.1610	5.6599
0.0076	31.01	16000	0.1571	5.1655
0.0181	33.01	17000	0.1449	5.4558
0.0522	35.0	18000	0.1340	5.8388
0.0356	37.0	19000	0.1458	5.0700
0.0132	39.0	20000	0.1310	5.1941

Framework versions

Transformers 4.33.0.dev0
PyTorch 2.0.1+cu117
Datasets 2.14.4
Tokenizers 0.13.3

How to use

from transformers import pipeline

hf_model = "HiTZ/whisper-large-ca"  # replace with actual repo ID
device = 0  # set to -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device
)

result = pipe("audio.wav")
print(result["text"])

Ethical considerations and risks

This model transcribes speech and may process personal data.
Users should ensure compliance with applicable data protection laws (e.g., GDPR).
The model should not be used for surveillance or non-consensual audio processing.

Citation

If you use this model in your research, please cite:

@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Please, check the related paper preprint in arXiv:2503.23542 for more details.

License

This model is available under the Apache-2.0 License. You are free to use, modify, and distribute this model as long as you credit the original creators.

Contact and attribution

Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
Base model: OpenAI Whisper
Dataset: Mozilla Common Voice

For questions or issues, please open an issue in the model repository.