|
--- |
|
tags: |
|
- text-to-speech |
|
license: cc-by-nc-sa-4.0 |
|
language: |
|
- zh |
|
- en |
|
- de |
|
- ja |
|
- fr |
|
- es |
|
- ko |
|
- ar |
|
- nl |
|
- ru |
|
- it |
|
- pl |
|
- pt |
|
pipeline_tag: text-to-speech |
|
inference: false |
|
--- |
|
|
|
|
|
This is a reformat of the [official Fish Speech v1.5 weights](https://huggingface.co/fishaudio/fish-speech-1.5) to work with [fish-speech.rs](https://github.com/EndlessReform/fish-speech.rs/tree/main). |
|
|
|
I've made the following changes, for better compatibility with [Candle.rs](https://github.com/huggingface/candle) and the HuggingFace ecosystem: |
|
- DualAR transformer weights converted to `.safetensors` for safety and easier loading |
|
- Tokenizer ported from [Tiktoken](https://github.com/openai/tiktoken) format and [custom wrapper](https://github.com/fishaudio/fish-speech/pull/696/files#diff-0db124559777c2399fda2a45cff039afa7eb1a53f2f98f628b02862c10b339ef) to HuggingFace Tokenizers for easier downstream use |
|
- VQGAN is unchanged from v1.4, so copying the weight-norm merged safetensors and FireflyGAN config from [my previous conversion](https://huggingface.co/jkeisling/fish-speech-1.4) |
|
|
|
NOTE: |
|
- Please respect the original license and do not use this model for commercial purposes. You can support Fish Audio by using the official API at [fish.audio](https://fish.audio/auth/). |
|
- These weights WILL NOT work with the [official Fish Speech inference code](https://github.com/fishaudio/fish-speech)! |
|
|
|
--- |
|
# ORIGINAL README: Fish Speech V1.5 |
|
|
|
**Fish Speech V1.5** is a leading text-to-speech (TTS) model trained on more than 1 million hours of audio data in multiple languages. |
|
|
|
Supported languages: |
|
- English (en) >300k hours |
|
- Chinese (zh) >300k hours |
|
- Japanese (ja) >100k hours |
|
- German (de) ~20k hours |
|
- French (fr) ~20k hours |
|
- Spanish (es) ~20k hours |
|
- Korean (ko) ~20k hours |
|
- Arabic (ar) ~20k hours |
|
- Russian (ru) ~20k hours |
|
- Dutch (nl) <10k hours |
|
- Italian (it) <10k hours |
|
- Polish (pl) <10k hours |
|
- Portuguese (pt) <10k hours |
|
|
|
Please refer to [Fish Speech Github](https://github.com/fishaudio/fish-speech) for more info. |
|
Demo available at [Fish Audio](https://fish.audio/). |
|
|
|
## Citation |
|
|
|
If you found this repository useful, please consider citing this work: |
|
|
|
``` |
|
@misc{fish-speech-v1.4, |
|
title={Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesis}, |
|
author={Shijia Liao and Yuxuan Wang and Tianyu Li and Yifan Cheng and Ruoyi Zhang and Rongzhi Zhou and Yijin Xing}, |
|
year={2024}, |
|
eprint={2411.01156}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.SD}, |
|
url={https://arxiv.org/abs/2411.01156}, |
|
} |
|
``` |
|
|
|
## License |
|
|
|
This model is permissively licensed under the BY-CC-NC-SA-4.0 license. |
