Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Deep neural networks for voice conversion (voice style transfer) in Tensorflow
| Date | Stars |
|---|---|
| 2026-07-31 | 3938 |
| 2026-08-03 | 3938 |
| 2026-08-05 | 3938 |
| 2026-08-06 | 3938 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Voice Conversion with Non-Parallel Data ## Subtitle: Speaking like Kate Winslet > Authors: Dabi Ahn([email protected]), [Kyubyong Park](https://github.com/Kyubyong)([email protected]) ## Samples https://soundcloud.com/andabi/sets/voice-style-transfer-to-kate-winslet-with-deep-neural-networks ## Intro What if you could imitate a famous celebrity's voice or sing like a famous singer? This project started with a goal to convert someone's voice to a specific target voice. So called, it's voice style transfer. We worked on this project that aims to convert someone's voice to a famous English actress [Kate Winslet](https://en.wikipedia.org/wiki/Kate_Winslet)'s [voice](https://soundcloud.com/andabi/sets/voice-style-transfer-to-kate-winslet-with-deep-neural-networks). We implemented a deep neural networks to achieve that and more than 2 hours of audio book sentences read by Kate Winslet are used as a dataset. <p align="center"><img src="https://raw.githubusercontent.com/andabi/deep-voice-conversion/master/materials/title.png" width="50%"></p> ## Model Architecture This is a many-to-one voice conversion system. The main significance of this work is that we could generate a target speaker's utterances without parallel data like <source's wav, target's wav>, <wav, text> or <wav, phone>, but only waveforms of the target speaker. (To make these parallel datasets needs a lot of effort.) All we need in this project is a number of waveforms of the target speaker's utterances and only a small set of <wav, phone> pairs from a number of anonymous speakers. <p align="center"><img src="https://raw.githubusercontent.com/andabi/deep-voice-conversion/master/materials/architecture.png" width="85%"></p> The model architecture consists of two modules: 1. Net1(phoneme classification) classify someone's utterances to one of phoneme classes at every timestep. * Phonemes are speaker-independent while waveforms are speaker-dependent. 2. Net2(speech synthesis) synthesize speeches of the target speaker from the phones. We applied CBHG(1-D convolution bank + highway network + bidirectional GRU) modules that are mentioned in [Tacotron](https://arxiv.org/abs/1703.10135). CBHG is known to be good for capturing features from sequential data. ### Net1 is a classifier. * Process: wav -> spectrogram -> mfccs -> phoneme dist. * Net1 classifies spectrogram to phonemes that consists of 60 English phonemes at every timestep. * For each timestep, the input is log magnitude spectrogram and the target is phoneme dist. * Objective function is cross entropy loss. * [TIMIT dataset](https://catalog.ldc.upenn.edu/LDC93S1) used. * contains 630 speakers' utterances and corresponding phones that speaks similar sentences. * Over 70% test accuracy ### Net2 is a synthesizer. Net2 contains Net1 as a sub-network. * Process: net1(wav -> spectrogram -> mfccs -> phoneme dist.) -> spectrogram -> wav * Net2 synthesizes the target speaker's speeches. * The input/target is a set of target speaker's utterances. * Since Net1 is already trained in previous step, the remaining part only should be trained in this step. * Loss is reconstruction error between input and target. (L2 distance) * Datasets * Target1(anonymous female): [Arctic](http://www.festvox.org/cmu_arctic/) dataset (public) * Target2(Kate Winslet): over 2 hours of audio book sentences read by her (private) * Griffin-Lim reconstruction when reverting wav from spectrogram. ## Implementations ### Requirements * python 2.7 * tensorflow >= 1.1 * numpy >= 1.11.1 * librosa == 0.5.1 ### Settings * sample rate: 16,000Hz * window length: 25ms * hop length: 5ms ### Procedure * Train phase: Net1 and Net2 should be trained sequentially. * Train1(training Net1) * Run `train1.py` to train and `eval1.py` to test. * Train2(training Net2) * Run `train2.py` to train and `eval2.py` to test. * Train2 should be trained after
Excerpt of 5,585 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:0bb4aae8824a88b3, llm:Description: 'Deep neural networks for voice conversion (voice style transfer) in Tensorflow'
matched fp:0bb4aae8824a88b3, llm:Description: 'Deep neural networks for voice conversion (voice style transfer) in Tensorflow'
matched fp:0bb4aae8824a88b3, llm:Description: 'Deep neural networks for voice conversion (voice style transfer) in Tensorflow'
