Upload Eval

README.md

@@ -23,10 +23,10 @@ model-index:
       args: pa-IN         # Optional. Example: zh-CN
     metrics:
       - type: wer    # Required. Example: wer
-        value: 39.47 # Required. Example: 20.90
-        name: Test WER    # Optional. Example: Test WER
+        value: 36.02 # Required. Example: 20.90
+        name: Test WER With LM    # Optional. Example: Test WER
       - type: cer    # Required. Example: wer
-        value: 13.60  # Required. Example: 20.90
+        value: 12.81 With LM  # Required. Example: 20.90
         name: Test CER    # Optional. Example: Test WER
 
 ---
@@ -42,7 +42,32 @@ It achieves the following results on the evaluation set:
 - Wer: 0.4939
 - Cer: 0.2238
 
+#### Evaluation Commands
+1. To evaluate on `mozilla-foundation/common_voice_8_0` with split `test`
 
+```bash
+python eval.py --model_id kingabzpro/wav2vec2-large-xlsr-53-punjabi --dataset mozilla-foundation/common_voice_8_0 --config pa-IN --split test
+```
+
+### Inference With LM
+
+```python
+import torch
+from datasets import load_dataset
+from transformers import AutoModelForCTC, AutoProcessor
+import torchaudio.functional as F
+model_id = "kingabzpro/wav2vec2-large-xlsr-53-punjabi"
+sample_iter = iter(load_dataset("mozilla-foundation/common_voice_8_0", "pa-IN", split="test", streaming=True, use_auth_token=True))
+sample = next(sample_iter)
+resampled_audio = F.resample(torch.tensor(sample["audio"]["array"]), 48_000, 16_000).numpy()
+model = AutoModelForCTC.from_pretrained(model_id)
+processor = AutoProcessor.from_pretrained(model_id)
+input_values = processor(resampled_audio, return_tensors="pt").input_values
+with torch.no_grad():
+    logits = model(input_values).logits
+transcription = processor.batch_decode(logits.numpy()).text
+
+```
 
 ### Training hyperparameters