Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A Non-Autoregressive Transformer based Text-to-Speech, supporting a family of SOTA transformers with supervised and unsupervised duration modelings. This project grows with the research community, aiming to achieve the ultimate TTS
| Date | Stars |
|---|---|
| 2026-07-24 | 328 |
| 2026-07-25 | 328 |
| 2026-07-28 | 328 |
| 2026-07-30 | 328 |
| 2026-07-31 | 328 |
| 2026-08-06 | 328 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Comprehensive-Transformer-TTS - PyTorch Implementation
**A Non-Autoregressive Transformer** based TTS, supporting a family of SOTA transformers with supervised and unsupervised duration modelings. This project grows with the research community, **aiming to achieve the ultimate TTS**. Any suggestions toward the best Non-AR TTS are welcome :)
### Transformers
- [x] [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084) (Wu et al., 2021)
- [x] [Long-Short Transformer: Efficient Transformers for Language and Vision](https://arxiv.org/abs/2107.02192) (Zhu et al., 2021)
- [x] [Conformer: Convolution-augmented Transformer for Speech Recognition](https://arxiv.org/abs/2005.08100) (Gulati et al., 2020)
- [x] [Reformer: The Efficient Transformer](https://arxiv.org/abs/2001.04451) (Kitaev et al., 2020)
- [x] [Attention Is All You Need](https://arxiv.org/abs/1706.03762) (Vaswani et al., 2017)
### Prosody Modelings (WIP)
- [x] [DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021](https://arxiv.org/abs/2110.12612) (Liu et al., 2021)
- [x] [Rich Prosody Diversity Modelling with Phone-level Mixture Density Network](https://arxiv.org/abs/2102.00851) (Du et al., 2021)
### Supervised Duration Modelings
- [x] [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://arxiv.org/abs/2006.04558) (Ren et al., 2020)
### Unsupervised Duration Modelings
- [x] [One TTS Alignment To Rule Them All](https://arxiv.org/abs/2108.10447) (Badlani et al., 2021): We are finally freed from external aligners such as MFA! Validation alignments for LJ014-0329 up to 70K are shown below as an example.
<p align="center">
<img src="./img/LJ014-0329.gif" width="60%">
</p>
### Transformer Performance Comparison on LJSpeech (1 TITAN RTX 24G / 16 batch size)
| Model | Memory Usage | Training Time (1K steps) |
| --- | ----------- | ----- |
|Fastformer (lucidrains')|10531MiB / 24220MiB|4m 25s
|Fastformer (wuch15's)|10515MiB / 24220MiB|4m 45s
|Long-Short Transformer|10633MiB / 24220MiB|5m 26s
|Conformer|18903MiB / 24220MiB|7m 4s
|Reformer|10293MiB / 24220MiB|10m 16s
|Transformer|7909MiB / 24220MiB|4m 51s
|Transformer_fs2|11571MiB / 24220MiB|4m 53s
Toggle the type of building blocks by
```yaml
# In the model.yaml
block_type: "transformer_fs2" # ["transformer_fs2", "transformer", "fastformer", "lstransformer", "conformer", "reformer"]
```
Toggle the type of prosody modelings by
```yaml
# In the model.yaml
prosody_modeling:
model_type: "none" # ["none", "du2021", "liu2021"]
```
Toggle the type of duration modelings by
```yaml
# In the model.yaml
duration_modeling:
learn_alignment: True # True for unsupervised modeling, and False for supervised modeling
```
# Quickstart
***DATASET*** refers to the names of datasets such as `LJSpeech` and `VCTK` in the following documents.
## Dependencies
You can install the Python dependencies with
```
pip3 install -r requirements.txt
```
Also, `Dockerfile` is provided for `Docker` users.
## Inference
You have to download the [pretrained models](https://drive.google.com/drive/folders/1xEOVbv3PLfGX8EgEkzg1014c9h8QMxQ-?usp=sharing) and put them in `output/ckpt/DATASET/`. The models are trained under unsupervised duration modeling with "transformer_fs2" building block.
For a **single-speaker TTS**, run
```
python3 synthesize.py --text "YOUR_DESIRED_TEXT" --restore_step RESTORE_STEP --mode single --dataset DATASET
```
For a **multi-speaker TTS**, run
```
python3 synthesize.py --text "YOUR_DESIRED_TEXT" --speaker_id SPEAKER_ID --restore_step RESTORE_STEP --mode single --dataset DATASET
```
The dictionary of learned speakers can be found at `preprocessed_data/DATASET/speakers.json`, and the generated utterances will be put in `output/result/`.
## Batch Inference
Batch inference is also supported, try
```
python3 synthesize.py --source preprocessed_data/DATASET/val.txt --restore_step RESTORE_STEP --mode batch --dataset DATASET
```
to synthesize alExcerpt of 12,454 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:faf8fbccbffc312f, topic:text-to-speech, topic:tts, topic:speech-synthesis
matched fp:faf8fbccbffc312f, topic:deep-learning, topic:pytorch