mistralai

/

Voxtral-Mini-3B-2507

like 543

Follow

Mistral AI_ 11.9k

Hey @sovitrath - I can't access the notebook, but to confirm: are you using the TranscriptionRequest API? https://huggingface.co/mistralai/Voxtral-Mini-3B-2507#transcription

The results you're getting suggest you might be using the ChatCompletionRequest API, which is not recommended for ASR, but rather Chat only.

Hello @sanchit-gandhi
I am running the model locally and using apply_transcription_request function. Basically, the following is the logic.

def map_to_pred(batch):
    audio = batch['audio']

    path = f'temp.wav'
    torchaudio.save(path, torch.tensor([audio['array']]), audio['sampling_rate'])

    input_features = processor.apply_transcrition_request(
        language='en',
        audio='temp.wav',
        sampling_rate=audio['sampling_rate'],
        # return_tensors='pt',
        model_id=repo_id,
        # format=[audio['array']]
    ).to(device, dtype=torch.bfloat16)

    # batch['reference'] = processor.tokenizer._normalize(batch['text'])
    batch['reference'] = whisper_processor.tokenizer._normalize(batch['text'])
    # batch['reference'] = batch['text']

    with torch.no_grad():
        predicted_ids = model.generate(**input_features)
    transcription = processor.batch_decode(
        predicted_ids[:, input_features.input_ids.shape[1]:],
        skip_special_tokens=True
    )
    # print(transcription)
    # print('#'*50)
    # batch['prediction'] = processor.tokenizer._normalize(transcription[0]) # Prediction is a single list.
    batch['prediction'] = whisper_processor.tokenizer._normalize(transcription[0]) # Prediction is a single list.
    # batch['prediction'] = transcription[0] # Prediction is a single list.

    # print(f'Reference: {batch['reference']}')
    # print(f'Predictions: {batch['prediction']}')
    return batch

The notebook I linked above is public. Not sure why it is not accessible. Please let me know if there is any other to send the code.

@sanchit-gandhi Try without authuser=1 at the end :)
https://colab.research.google.com/drive/1Ob9DwqBvGDobLjiOT3zLJV8IL-Xohu9r

Hey there,

Here is a reproducer using the Transformers implementation, yielding 1.89% WER on librispeech clean test set 🤗
As can be found in the inference snippets and the doc, you need to set the max_new_tokens generation parameter 😉.

Indeed, in Transformers, we go with the convention of setting this parameter to 20 by default.

Thank you @eustlb
This helps a lot.