Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
| Date | Stars |
|---|---|
| 2026-07-24 | 6316 |
| 2026-07-25 | 6318 |
| 2026-07-28 | 6318 |
| 2026-07-30 | 6318 |
| 2026-07-31 | 6324 |
| 2026-08-07 | 6328 |
| 2026-08-15 | 6328 |
| 2026-08-18 | 6330 |
| 2026-08-19 | 6331 |
| 2026-08-20 | 6332 |
| 2026-08-21 | 6331 |
| 2026-08-23 | 6332 |
| 2026-08-24 | 6333 |
| 2026-08-25 | 6334 |
| 2026-08-26 | 6335 |
| 2026-08-27 | 6337 |
| 2026-08-29 | 6339 |
| 2026-08-31 | 6339 |
| 2026-09-01 | 6340 |
| 2026-09-02 | 6342 |
| 2026-09-03 | 6344 |
| 2026-09-04 | 6344 |
| 2026-09-05 | 6345 |
| 2026-09-06 | 6345 |
| 2026-09-07 | 6345 |
| 2026-09-09 | 6346 |
| 2026-09-10 | 6347 |
| 2026-09-11 | 6347 |
| 2026-09-12 | 6350 |
| 2026-09-13 | 6350 |
| 2026-09-14 | 6349 |
| 2026-09-15 | 6349 |
| 2026-09-16 | 6352 |
| 2026-09-17 | 6353 |
| 2026-09-18 | 6354 |
| 2026-09-19 | 6355 |
| 2026-09-20 | 6355 |
Today
— stars today
This week
+5 stars this week
This month
+24 stars this month
Momentum
0.0
growth rate 0.08%/day
# StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models ### Yinghao Aaron Li, Cong Han, Vinay S. Raghavan, Gavin Mischler, Nima Mesgarani > In this paper, we present StyleTTS 2, a text-to-speech (TTS) model that leverages style diffusion and adversarial training with large speech language models (SLMs) to achieve human-level TTS synthesis. StyleTTS 2 differs from its predecessor by modeling styles as a latent random variable through diffusion models to generate the most suitable style for the text without requiring reference speech, achieving efficient latent diffusion while benefiting from the diverse speech synthesis offered by diffusion models. Furthermore, we employ large pre-trained SLMs, such as WavLM, as discriminators with our novel differentiable duration modeling for end-to-end training, resulting in improved speech naturalness. StyleTTS 2 surpasses human recordings on the single-speaker LJSpeech dataset and matches it on the multispeaker VCTK dataset as judged by native English speakers. Moreover, when trained on the LibriTTS dataset, our model outperforms previous publicly available models for zero-shot speaker adaptation. This work achieves the first human-level TTS synthesis on both single and multispeaker datasets, showcasing the potential of style diffusion and adversarial training with large SLMs. Paper: [https://arxiv.org/abs/2306.07691](https://arxiv.org/abs/2306.07691) Audio samples: [https://styletts2.github.io/](https://styletts2.github.io/) Online demo: [Hugging Face](https://huggingface.co/spaces/styletts2/styletts2) (thank [@fakerybakery](https://github.com/fakerybakery) for the wonderful online demo) [](https://colab.research.google.com/github/yl4579/StyleTTS2/blob/main/) [](https://discord.gg/ha8sxdG2K4) ## TODO - [x] Training and inference demo code for single-speaker models (LJSpeech) - [x] Test training code for multi-speaker models (VCTK and LibriTTS) - [x] Finish demo code for multispeaker model and upload pre-trained models - [x] Add a finetuning script for new speakers with base pre-trained multispeaker models - [ ] Fix DDP (accelerator) for `train_second.py` **(I have tried everything I could to fix this but had no success, so if you are willing to help, please see [#7](https://github.com/yl4579/StyleTTS2/issues/7))** ## Pre-requisites 1. Python >= 3.7 2. Clone this repository: ```bash git clone https://github.com/yl4579/StyleTTS2.git cd StyleTTS2 ``` 3. Install python requirements: ```bash pip install -r requirements.txt ``` On Windows add: ```bash pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 -U ``` Also install phonemizer and espeak if you want to run the demo: ```bash pip install phonemizer sudo apt-get install espeak-ng ``` 4. Download and extract the [LJSpeech dataset](https://keithito.com/LJ-Speech-Dataset/), unzip to the data folder and upsample the data to 24 kHz. The text aligner and pitch extractor are pre-trained on 24 kHz data, but you can easily change the preprocessing and re-train them using your own preprocessing. For LibriTTS, you will need to combine train-clean-360 with train-clean-100 and rename the folder train-clean-460 (see [val_list_libritts.txt](https://github.com/yl4579/StyleTTS/blob/main/Data/val_list_libritts.txt) as an example). ## Training First stage training: ```bash accelerate launch train_first.py --config_path ./Configs/config.yml ``` Second stage training **(DDP version not working, so the current version uses DP, again see [#7](https://github.com/yl4579/StyleTTS2/issues/7) if you want to help)**: ```bash python train_second.py --config_path ./Configs/config.yml ``` You can run both consecutively and it will train both the first and se
Excerpt of 14,167 characters
Read on GitHub
93
5
3
David Wolf
2
Ikko Eltociear Ashimine · Japan
2
1
1
1
Ameer Azam · AICSC · India
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:84d5fe4d644a8419, topic:text-to-speech, topic:tts, topic:speech-synthesis
matched fp:84d5fe4d644a8419, topic:deep-learning, topic:pytorch
matched fp:84d5fe4d644a8419, topic:diffusion-models, topic:gan, readme:latent diffusion