Fine-tuned SpeechT5 TTS Model for Haitian Creole

This model is a fine-tuned version of microsoft/speecht5-tts for the Haitian Creole language. It was fine-tuned on the CMU Haitian dataset.

Model Description

The model is based on the SpeechT5 architecture, which is a variant of the T5 (Text-to-Text Transfer Transformer) model designed specifically for text-to-speech tasks. The model is capable of converting input text in Haitian Creole into corresponding speech.

Intended Uses & Limitations

The model is intended for text-to-speech (TTS) applications in Haitian Creole language processing. It can be used for generating speech from written text, enabling applications such as audiobook narration, voice assistants, and more.

However, there are some limitations to be aware of:

The model's performance heavily depends on the quality and diversity of the training data. Fine-tuning on more diverse and specific datasets might improve its performance.
Like all machine learning models, this model may produce inaccuracies or errors in speech synthesis, especially for complex sentences or domain-specific jargon.

Training and Evaluation Data

The model was fine-tuned on the CMU Haitian dataset, which contains text and corresponding audio samples in Haitian Creole. The dataset was split into training and evaluation sets to assess the model's performance.

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
per_device_train_batch_size: 16
gradient_accumulation_steps: 2
warmup_steps: 500
max_steps: 4000
gradient_checkpointing: True
fp16: True
evaluation_strategy: no
per_device_eval_batch_size: 8
save_steps: 1000
logging_steps: 25
report_to: ["tensorboard"]
greater_is_better: False

Training Results

The training progress and evaluation results are as follows:

Training Loss	Epoch	Step	Validation Loss
0.5147	2.42	1000	0.4753
0.4932	4.84	2000	0.4629
0.4926	7.26	3000	0.4566
0.4907	9.69	4000	0.4542
0.4839	12.11	5000	0.4532

Training Output

The training was completed with the following output:

Global Step: 4000
Training Loss: 0.3344
Training Runtime: 7123.63 seconds
Training Samples per Second: 17.97
Training Steps per Second: 0.562
Total FLOPs: 1.1690e+16

Framework Versions

Transformers 4.31.0
PyTorch 2.0.1+cu118
Datasets 2.13.1
Tokenizers 0.13.3