--- language: - he license: apache-2.0 base_model: openai/whisper-medium tags: - hf-asr-leaderboard - generated_from_trainer metrics: - wer model-index: - name: he results: [] --- # he This model is a fine-tuned version of [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.1297 - Wer: 11.0347 - Avg Precision Exact: 0.9146 - Avg Recall Exact: 0.9160 - Avg F1 Exact: 0.9149 - Avg Precision Letter Shift: 0.9325 - Avg Recall Letter Shift: 0.9340 - Avg F1 Letter Shift: 0.9329 - Avg Precision Word Level: 0.9345 - Avg Recall Word Level: 0.9358 - Avg F1 Word Level: 0.9348 - Avg Precision Word Shift: 0.9770 - Avg Recall Word Shift: 0.9786 - Avg F1 Word Shift: 0.9774 - Precision Median Exact: 1.0 - Recall Median Exact: 1.0 - F1 Median Exact: 1.0 - Precision Max Exact: 1.0 - Recall Max Exact: 1.0 - F1 Max Exact: 1.0 - Precision Min Exact: 0.0 - Recall Min Exact: 0.0 - F1 Min Exact: 0.0 - Precision Min Letter Shift: 0.0 - Recall Min Letter Shift: 0.0 - F1 Min Letter Shift: 0.0 - Precision Min Word Level: 0.0 - Recall Min Word Level: 0.0 - F1 Min Word Level: 0.0 - Precision Min Word Shift: 0.2222 - Recall Min Word Shift: 0.1667 - F1 Min Word Shift: 0.1905 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-05 - train_batch_size: 8 - eval_batch_size: 32 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - training_steps: 20000 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Wer | Avg Precision Exact | Avg Recall Exact | Avg F1 Exact | Avg Precision Letter Shift | Avg Recall Letter Shift | Avg F1 Letter Shift | Avg Precision Word Level | Avg Recall Word Level | Avg F1 Word Level | Avg Precision Word Shift | Avg Recall Word Shift | Avg F1 Word Shift | Precision Median Exact | Recall Median Exact | F1 Median Exact | Precision Max Exact | Recall Max Exact | F1 Max Exact | Precision Min Exact | Recall Min Exact | F1 Min Exact | Precision Min Letter Shift | Recall Min Letter Shift | F1 Min Letter Shift | Precision Min Word Level | Recall Min Word Level | F1 Min Word Level | Precision Min Word Shift | Recall Min Word Shift | F1 Min Word Shift | |:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------:|:------------:|:-------------------:|:----------------:|:------------:|:--------------------------:|:-----------------------:|:-------------------:|:------------------------:|:---------------------:|:-----------------:|:------------------------:|:---------------------:|:-----------------:| | 0.1333 | 0.08 | 1000 | 0.1862 | 26.0680 | 0.7918 | 0.7852 | 0.7876 | 0.8219 | 0.8148 | 0.8173 | 0.8262 | 0.8215 | 0.8229 | 0.9129 | 0.9174 | 0.9139 | 0.875 | 0.8667 | 0.8696 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1111 | 0.1 | 0.1176 | | 0.1035 | 0.16 | 2000 | 0.1392 | 19.0059 | 0.8428 | 0.8425 | 0.8421 | 0.8699 | 0.8700 | 0.8693 | 0.8735 | 0.8745 | 0.8734 | 0.9454 | 0.9492 | 0.9466 | 0.9167 | 0.9167 | 0.9167 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.0459 | 0.24 | 3000 | 0.1279 | 16.5115 | 0.8767 | 0.8820 | 0.8788 | 0.9011 | 0.9066 | 0.9033 | 0.9041 | 0.9098 | 0.9064 | 0.9603 | 0.9671 | 0.9631 | 0.9286 | 0.9375 | 0.9474 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1111 | 0.1111 | 0.125 | | 0.0453 | 0.32 | 4000 | 0.1197 | 15.1293 | 0.8892 | 0.8902 | 0.8892 | 0.9125 | 0.9137 | 0.9126 | 0.9154 | 0.9167 | 0.9155 | 0.9642 | 0.9657 | 0.9643 | 0.9375 | 0.9375 | 0.9524 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.0306 | 0.4 | 5000 | 0.1224 | 14.5344 | 0.8894 | 0.8918 | 0.8901 | 0.9134 | 0.9161 | 0.9143 | 0.9163 | 0.9190 | 0.9171 | 0.9656 | 0.9693 | 0.9669 | 0.9375 | 1.0 | 0.9565 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.0283 | 0.48 | 6000 | 0.1170 | 13.7029 | 0.8910 | 0.8903 | 0.8902 | 0.9112 | 0.9104 | 0.9103 | 0.9139 | 0.9139 | 0.9134 | 0.9673 | 0.9692 | 0.9677 | 1.0 | 1.0 | 0.9655 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0211 | 0.56 | 7000 | 0.1234 | 13.1559 | 0.8945 | 0.8973 | 0.8954 | 0.9145 | 0.9174 | 0.9154 | 0.9170 | 0.9199 | 0.9180 | 0.9683 | 0.9716 | 0.9694 | 1.0 | 1.0 | 0.9677 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.0228 | 0.64 | 8000 | 0.1265 | 13.5070 | 0.8960 | 0.8960 | 0.8955 | 0.9140 | 0.9140 | 0.9135 | 0.9164 | 0.9169 | 0.9162 | 0.9675 | 0.9710 | 0.9687 | 1.0 | 1.0 | 0.9655 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1 | 0.1176 | | 0.0095 | 0.72 | 9000 | 0.1269 | 12.9527 | 0.8969 | 0.8982 | 0.8972 | 0.9179 | 0.9194 | 0.9182 | 0.9205 | 0.9218 | 0.9207 | 0.9702 | 0.9727 | 0.9710 | 1.0 | 1.0 | 0.9677 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1111 | 0.125 | | 0.009 | 0.8 | 10000 | 0.1270 | 12.6053 | 0.9003 | 0.9046 | 0.9020 | 0.9201 | 0.9246 | 0.9219 | 0.9224 | 0.9267 | 0.9241 | 0.9710 | 0.9754 | 0.9727 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0053 | 0.88 | 11000 | 0.1274 | 12.4427 | 0.8978 | 0.9016 | 0.8993 | 0.9174 | 0.9213 | 0.9190 | 0.9192 | 0.9232 | 0.9208 | 0.9699 | 0.9744 | 0.9717 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.125 | 0.125 | 0.1333 | | 0.0203 | 0.96 | 12000 | 0.1270 | 12.1471 | 0.9077 | 0.9098 | 0.9084 | 0.9271 | 0.9293 | 0.9278 | 0.9298 | 0.9320 | 0.9305 | 0.9729 | 0.9752 | 0.9736 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0133 | 1.04 | 13000 | 0.1289 | 12.1212 | 0.9072 | 0.9075 | 0.9070 | 0.9265 | 0.9268 | 0.9263 | 0.9289 | 0.9294 | 0.9288 | 0.9744 | 0.9751 | 0.9743 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0833 | 0.1111 | 0.0952 | | 0.0055 | 1.12 | 14000 | 0.1282 | 11.6667 | 0.9090 | 0.9120 | 0.9101 | 0.9275 | 0.9306 | 0.9286 | 0.9295 | 0.9323 | 0.9305 | 0.9745 | 0.9780 | 0.9758 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.007 | 1.2 | 15000 | 0.1254 | 11.6888 | 0.9075 | 0.9097 | 0.9082 | 0.9260 | 0.9282 | 0.9267 | 0.9282 | 0.9302 | 0.9288 | 0.9735 | 0.9760 | 0.9743 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1 | 0.1176 | | 0.0122 | 1.28 | 16000 | 0.1304 | 11.5336 | 0.9128 | 0.9133 | 0.9127 | 0.9314 | 0.9320 | 0.9313 | 0.9336 | 0.9338 | 0.9333 | 0.9767 | 0.9773 | 0.9766 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.2222 | 0.1667 | 0.1905 | | 0.0041 | 1.36 | 17000 | 0.1309 | 11.3267 | 0.9126 | 0.9141 | 0.9130 | 0.9304 | 0.9319 | 0.9308 | 0.9327 | 0.9340 | 0.9330 | 0.9750 | 0.9770 | 0.9756 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1 | 0.1176 | | 0.0139 | 1.44 | 18000 | 0.1278 | 11.1678 | 0.9141 | 0.9140 | 0.9137 | 0.9321 | 0.9319 | 0.9316 | 0.9337 | 0.9335 | 0.9333 | 0.9771 | 0.9773 | 0.9768 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.1 | 0.1176 | | 0.001 | 1.52 | 19000 | 0.1294 | 11.2232 | 0.9135 | 0.9146 | 0.9137 | 0.9319 | 0.9331 | 0.9321 | 0.9338 | 0.9348 | 0.9340 | 0.9763 | 0.9775 | 0.9765 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.1429 | 0.125 | 0.1333 | | 0.0068 | 1.6 | 20000 | 0.1297 | 11.0347 | 0.9146 | 0.9160 | 0.9149 | 0.9325 | 0.9340 | 0.9329 | 0.9345 | 0.9358 | 0.9348 | 0.9770 | 0.9786 | 0.9774 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.2222 | 0.1667 | 0.1905 | ### Framework versions - Transformers 4.39.0.dev0 - Pytorch 2.2.1+cu121 - Datasets 2.16.1 - Tokenizers 0.15.0