Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
PyTorch Implementation of ProDiff (ACM-MM'22) with a Extremely-Fast diffusion speech synthesis pipeline
| Date | Stars |
|---|---|
| 2026-07-24 | 432 |
| 2026-07-25 | 432 |
| 2026-07-28 | 432 |
| 2026-07-30 | 432 |
| 2026-08-06 | 432 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech
#### Rongjie Huang, Zhou Zhao, Huadai Liu, Jinglin Liu, Chenye Cui, Yi Ren
PyTorch Implementation of [ProDiff (ACM Multimedia'22)](https://arxiv.org/abs/2207.06389): a conditional diffusion probabilistic model capable of generating high fidelity speech efficiently.
[](https://arxiv.org/abs/2207.06389)
[](https://github.com/Rongjiehuang/ProDiff)
[](https://huggingface.co/spaces/Rongjiehuang/ProDiff)
We provide our implementation and pretrained models as open source in this repository.
Visit our [demo page](https://prodiff.github.io/) for audio samples.
## News
- April, 2022: Our previous work **[FastDiff](https://arxiv.org/abs/2204.09934) (IJCAI 2022)** released in [Github](https://github.com/Rongjiehuang/FastDiff).
- September, 2022: **[ProDiff](https://arxiv.org/abs/2207.06389) (ACM Multimedia 2022)** released in Github.
## Key Features
- **Extremely-Fast** diffusion text-to-speech synthesis pipeline for potential **industrial deployment**.
- **Tutorial and code base** for speech diffusion models.
- More **supported diffusion mechanism** (e.g., guided diffusion) will be available.
## Quick Started
We provide an example of how you can generate high-fidelity samples using ProDiff.
To try on your own dataset, simply clone this repo in your local machine provided with NVIDIA GPU + CUDA cuDNN and follow the below instructions.
### Support Datasets and Pretrained Models
Simply run following command to download the weights
```python
from huggingface_hub import snapshot_download
downloaded_path = snapshot_download(repo_id="Rongjiehuang/ProDiff")
```
and move the downloaded checkpoints to `checkpoints/$Model/model_ckpt_steps_*.ckpt`
```bash
mv ${downloaded_path}/checkpoints/ checkpoints/
```
Details of each folder are as in follows:
| Model | Dataset | Config |
|-------------------|-------------|-------------------------------------------------|
| ProDiff Teacher | LJSpeech | `modules/ProDiff/config/prodiff_teacher.yaml` |
| ProDiff | LJSpeech | `modules/ProDiff/config/prodiff.yaml` |
More supported datasets are coming soon.
### Dependencies
See requirements in `requirement.txt`:
- [pytorch](https://github.com/pytorch/pytorch)
- [librosa](https://github.com/librosa/librosa)
- [NATSpeech](https://github.com/NATSpeech/NATSpeech)
### Multi-GPU
By default, this implementation uses as many GPUs in parallel as returned by `torch.cuda.device_count()`.
You can specify which GPUs to use by setting the `CUDA_DEVICES_AVAILABLE` environment variable before running the training module.
## Extremely-Fast Text-to-Speech with diffusion probabilistic models
Here we provide a speech synthesis pipeline using diffusion probabilistic models: ProDiff (acoustic model) + FastDiff (neural vocoder). [](https://huggingface.co/spaces/Rongjiehuang/ProDiff)
1. Prepare acoustic model (ProDiff or ProDiff Teacher): Download LJSpeech checkpoint and put it in `checkpoints/ProDiff` or `checkpoints/ProDiff_Teacher`
2. Prepare neural vocoder (FastDiff): Download LJSpeech checkpoint and put it in `checkpoints/FastDiff`
3. Specify the input `$text`, and set `N` for reverse sampling in neural vocoder, which is a trade off between quality and speed.
4. Run the following command for extreme fast speed `(2-iter ProDiff + 4-iter FastDiff)`:
```bash
CUDA_VISIBLE_DEVICES=$GPU python inference/ProDiff.py --config modules/ProDiff/config/prodiff.yaml --exp_name ProDiff --hparams="N=4,text='$txt'" --resetExcerpt of 7,690 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:e2e1fd0686a5e908, topic:text-to-speech, topic:speech-synthesis, readme:text-to-speech