ttsds/results
Viewer
•
Updated
•
2
•
86
•
1
Christoph Minixhofer PRO
cdminix
AI & ML interests
None yet
Recent Activity
New activity
2 days ago
ttsds/benchmark:The .txt file when submitting dataset
updated
a dataset
5 days ago
ttsds/results
New activity
5 days ago
Pendrokar/open_tts_tracker:merging with ttsds/datasets
Organizations
cdminix's activity
The .txt file when submitting dataset
1
#4 opened 3 days ago
by
ecyht2
ecyht2
updated
a
dataset
5 days ago
merging with ttsds/datasets
3
#4 opened 5 days ago
by
cdminix
cdminix
posted
an
update
5 days ago
Post
955
As part of some ongoing work, I'm releasing the currently biggest collection of docker containers for state-of-the-art voice cloning TTS systems.
https://github.com/ttsds/datasets
Alongside there is also a nice overview of all systems (see below)
https://github.com/ttsds/datasets
Alongside there is also a nice overview of all systems (see below)
updated
a
dataset
5 days ago
updated
a
model
24 days ago
Synthetic speech
4
#6 opened about 1 month ago
by
cdminix
cdminix
liked
2
Spaces
about 1 month ago
upvoted
a
collection
about 2 months ago
updated
a
dataset
2 months ago
updated
a
Space
3 months ago
updated
a
dataset
3 months ago
[MODELS] suggest new models here
5
#2 opened 4 months ago
by
cdminix
cdminix
updated
a
Space
4 months ago
liked
a
model
4 months ago
replied to
their
post
4 months ago
Totally agree! Tortoise seems to not get benchmarked/compared to as much as other systems, and I don't know exactly why.
Not just for Tortoise, but for all theses systems it would be interesting how they compare to each other when finetuned. Unfortantely I don't know of any benchmarks/papers that have tried to evaluate that (yet).
posted
an
update
4 months ago
Post
503
I just added 5 more models to my open source TTS model benchmark,
ttsds/benchmark.
Let's talk about the results!
Over the last couple days, I added jbetker/tortoise-tts-v2, metavoiceio/metavoice-1B-v0.1, audo/HierSpeechpp, and the unofficial implementations of amphion/NaturalSpeech2 and amphion/valle by https://huggingface.co/amphion
Takeaways:
- TorToiSe does very well, falling into second place after StyleTTS 2, which is also ranked first in the human evaluation at TTS-AGI/TTS-Arena.
- MetaVoice-1B's overall score is dragged down by its Intelligibility Score (probably due to utterances being cut short), it achieves #3 in Speaker Score, which indicates good voice cloning ability.
- HierSpeech++ lands in the middle of the road in terms of performance, but excels at the Environment Score, achieving #2 - this means the model is especially good at modeling recording conditions such as microphone and background noise.
- The Amphion models, possibly due to not being trained for the same amount as in the papers, achieve relatively low scores. However, they seem to struggle for different reasons. The autoregressive VALLE models have low Intelligibility Scores (possibly due to "babbling" or early stop tokens) while NaturalSpeech2 has low Speaker and Prosody scores.
What's next?
I'm planning to add more open source TTS models like suno/bark, CAMB-AI/MARS5-TTS and fishaudio/fish-speech-1.2. I'll also write an article on these and all the other results soon, since our paper, TTSDS -- Text-to-Speech Distribution Score (2407.12707), mostly focused on establishing the benchmark itself rather than the indiviual TTS systems.
Let's talk about the results!
Over the last couple days, I added jbetker/tortoise-tts-v2, metavoiceio/metavoice-1B-v0.1, audo/HierSpeechpp, and the unofficial implementations of amphion/NaturalSpeech2 and amphion/valle by https://huggingface.co/amphion
Takeaways:
- TorToiSe does very well, falling into second place after StyleTTS 2, which is also ranked first in the human evaluation at TTS-AGI/TTS-Arena.
- MetaVoice-1B's overall score is dragged down by its Intelligibility Score (probably due to utterances being cut short), it achieves #3 in Speaker Score, which indicates good voice cloning ability.
- HierSpeech++ lands in the middle of the road in terms of performance, but excels at the Environment Score, achieving #2 - this means the model is especially good at modeling recording conditions such as microphone and background noise.
- The Amphion models, possibly due to not being trained for the same amount as in the papers, achieve relatively low scores. However, they seem to struggle for different reasons. The autoregressive VALLE models have low Intelligibility Scores (possibly due to "babbling" or early stop tokens) while NaturalSpeech2 has low Speaker and Prosody scores.
What's next?
I'm planning to add more open source TTS models like suno/bark, CAMB-AI/MARS5-TTS and fishaudio/fish-speech-1.2. I'll also write an article on these and all the other results soon, since our paper, TTSDS -- Text-to-Speech Distribution Score (2407.12707), mostly focused on establishing the benchmark itself rather than the indiviual TTS systems.
-
- 3 replies
[MODELS] suggest new models here
5
#2 opened 4 months ago
by
cdminix
cdminix
updated
a
dataset
4 months ago