large-v2-multiple-bg-v1

This model is a fine-tuned version of openai/whisper-large-v2 on the audiofolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.8192

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.01
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.7975	17.2414	500	1.7851
1.4044	34.4828	1000	1.3945
1.0932	51.7241	1500	1.0965
0.9793	68.9655	2000	0.9987
0.9105	86.2069	2500	0.9390
0.8601	103.4483	3000	0.8936
0.8238	120.6897	3500	0.8600
0.7935	137.9310	4000	0.8370
0.7834	155.1724	4500	0.8236
0.7751	172.4138	5000	0.8192

Framework versions

• PEFT 0.10.0
• Transformers 4.41.0.dev0
• Pytorch 2.2.2+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1