---
library_name: peft
language:
- sr
license: apache-2.0
base_model: openai/whisper-large-v2
tags:
- generated_from_trainer
model-index:
- name: Whisper - Serbian Model
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Whisper - Serbian Model

This model is a fine-tuned version of [openai/whisper-large-v2](https://huggingface.co/openai/whisper-large-v2) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1118

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0009
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 400
- training_steps: 16000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step  | Validation Loss |
|:-------------:|:------:|:-----:|:---------------:|
| 0.2185        | 0.0705 | 500   | 0.2191          |
| 0.1764        | 0.1409 | 1000  | 0.1864          |
| 0.1752        | 0.2114 | 1500  | 0.1786          |
| 0.1703        | 0.2819 | 2000  | 0.1690          |
| 0.1558        | 0.3524 | 2500  | 0.1629          |
| 0.1584        | 0.4228 | 3000  | 0.1586          |
| 0.1496        | 0.4933 | 3500  | 0.1547          |
| 0.1592        | 0.5638 | 4000  | 0.1520          |
| 0.1428        | 0.6342 | 4500  | 0.1496          |
| 0.1429        | 0.7047 | 5000  | 0.1459          |
| 0.1357        | 0.7752 | 5500  | 0.1443          |
| 0.1332        | 0.8457 | 6000  | 0.1406          |
| 0.1413        | 0.9161 | 6500  | 0.1398          |
| 0.1373        | 0.9866 | 7000  | 0.1369          |
| 0.1253        | 1.0571 | 7500  | 0.1345          |
| 0.1229        | 1.1276 | 8000  | 0.1330          |
| 0.1114        | 1.1980 | 8500  | 0.1316          |
| 0.1189        | 1.2685 | 9000  | 0.1292          |
| 0.1116        | 1.3390 | 9500  | 0.1266          |
| 0.1108        | 1.4094 | 10000 | 0.1258          |
| 0.1098        | 1.4799 | 10500 | 0.1243          |
| 0.1112        | 1.5504 | 11000 | 0.1223          |
| 0.1139        | 1.6209 | 11500 | 0.1212          |
| 0.0951        | 1.6913 | 12000 | 0.1200          |
| 0.108         | 1.7618 | 12500 | 0.1173          |
| 0.104         | 1.8323 | 13000 | 0.1156          |
| 0.0963        | 1.9027 | 13500 | 0.1146          |
| 0.1004        | 1.9732 | 14000 | 0.1133          |
| 0.0831        | 2.0437 | 14500 | 0.1131          |
| 0.0779        | 2.1142 | 15000 | 0.1126          |
| 0.0805        | 2.1846 | 15500 | 0.1121          |
| 0.0849        | 2.2551 | 16000 | 0.1118          |


### Framework versions

- PEFT 0.13.2
- Transformers 4.46.3
- Pytorch 2.5.1
- Datasets 3.0.0
- Tokenizers 0.20.3