Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
PyTorch Implementation of GenerSpeech (NeurIPS'22): a text-to-speech model towards zero-shot style transfer of OOD custom voice.
| Date | Stars |
|---|---|
| 2026-07-24 | 333 |
| 2026-07-25 | 333 |
| 2026-07-28 | 333 |
| 2026-07-30 | 333 |
| 2026-07-31 | 333 |
| 2026-08-06 | 333 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech #### Rongjie Huang, Yi Ren, Jinglin Liu, Chenye Cui, Zhou Zhao | Zhejiang University, Sea AI Lab PyTorch Implementation of [GenerSpeech (NeurIPS'22)](https://arxiv.org/abs/2205.07211): a text-to-speech model towards high-fidelity zero-shot style transfer of OOD custom voice. [](https://arxiv.org/abs/2205.07211) [](https://github.com/Rongjiehuang/GenerSpeech) We provide our implementation and pretrained models in this repository. Visit our [demo page](https://generspeech.github.io/) for audio samples. ## News - December, 2022: **[GenerSpeech](https://arxiv.org/abs/2205.07211) (NeurIPS 2022)** released at Github. ## Key Features - **Multi-level Style Transfer** for expressive text-to-speech. - **Enhanced model generalization** to out-of-distribution (OOD) style reference. ## Quick Started We provide an example of how you can generate high-fidelity samples using GenerSpeech. To try on your own dataset, simply clone this repo in your local machine provided with NVIDIA GPU + CUDA cuDNN and follow the below instructions. ### Support Datasets and Pretrained Models You can use pretrained models we provide [here](https://huggingface.co/spaces/Rongjiehuang/GenerSpeech/tree/main/checkpoints), and data [here](https://huggingface.co/spaces/Rongjiehuang/GenerSpeech/tree/main/data/binary/training_set). Details of each folder are as in follows: | Model | Dataset (16 kHz) | Discription | |-------------|------------------|--------------------------------------------------------------------------| | GenerSpeech | LibriTTS,ESD | Acousitic model [(config)](modules/GenerSpeech/config/generspeech.yaml) | | HIFI-GAN | LibriTTS,ESD | Neural Vocoder | | Encoder | / | Emotion Encoder | More supported datasets are coming soon. ### Dependencies A suitable [conda](https://conda.io/) environment named `generspeech` can be created and activated with: ``` conda env create -f environment.yaml conda activate generspeech ``` ### Multi-GPU By default, this implementation uses as many GPUs in parallel as returned by `torch.cuda.device_count()`. You can specify which GPUs to use by setting the `CUDA_DEVICES_AVAILABLE` environment variable before running the training module. ## Inference (Zero-shot TTS) Here we provide a speech synthesis pipeline using GenerSpeech. 1. Prepare **GenerSpeech** (acoustic model): Download and put checkpoint at `checkpoints/GenerSpeech` 2. Prepare **HIFI-GAN** (neural vocoder): Download and put checkpoint at `checkpoints/trainset_hifigan` 3. Prepare **Emotion Encoder**: Download and put checkpoint at `checkpoints/Emotion_encoder.pt` 4. Prepare **dataset**: Download and put statistical files at `data/binary/training_set` 5. Prepare **path/to/reference_audio (16k)**: By default, GenerSpeech uses **[ASR](https://huggingface.co/facebook/wav2vec2-base-960h) + [MFA](https://montreal-forced-aligner.readthedocs.io/)** to obtain the text-speech alignment from reference. ```bash CUDA_VISIBLE_DEVICES=$GPU python inference/GenerSpeech.py --config modules/GenerSpeech/config/generspeech.yaml --exp_name GenerSpeech --hparams="text='here we go',ref_audio='assets/0011_001570.wav'" ``` Generated wav files are saved in `infer_out` by default.<br> # Train your own model ### Data Preparation and Configuration ## 1. Set `raw_data_dir`, `processed_data_dir`, `binary_data_dir` in the config file, and download dataset to `raw_data_dir`. 2. Check `preprocess_cls` in the config file. The dataset structure needs to follow the processor `preprocess_cls`, or you could rewrite it according to your dataset. We
Excerpt of 6,129 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:a3e6e671a737543d, topic:text-to-speech, topic:tts, topic:speech-synthesis