Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Voice Conversion With Just Nearest Neighbors
| Date | Stars |
|---|---|
| 2026-07-24 | 520 |
| 2026-07-25 | 520 |
| 2026-07-28 | 520 |
| 2026-07-30 | 520 |
| 2026-08-06 | 520 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Voice Conversion With Just Nearest Neighbors (kNN-VC)
The official code repo! This repo contains training and inference code for kNN-VC -- an any-to-any voice conversion model from our paper, "Voice Conversion With Just k-Nearest Neighbors". The trained checkpoints are available under the 'Releases' tab and through `torch.hub`.
Links:
- Arxiv paper: [https://arxiv.org/abs/2305.18975](https://arxiv.org/abs/2305.18975)
- Colab quickstart: <a target="_blank" href="https://colab.research.google.com/github/bshall/knn-vc/blob/master/knnvc_demo.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
- Interspeech proceedings: [https://www.isca-speech.org/archive/interspeech_2023/baas23_interspeech.html](https://www.isca-speech.org/archive/interspeech_2023/baas23_interspeech.html)
- Demo page with samples: [https://bshall.github.io/knn-vc/](https://bshall.github.io/knn-vc/)
Figure: kNN-VC setup. The source and reference utterance(s) are encoded into self-supervised features using WavLM. Each source feature is assigned to the mean of the k closest features from the reference. The resulting feature sequence is then vocoded with HiFi-GAN to arrive at the converted waveform output.
**Authors**:
- [Matthew Baas](https://rf5.github.io/)*
- [Benjamin van Niekerk](https://scholar.google.com/citations?user=zCokvy8AAAAJ&hl=en&oi=ao)*
- [Herman Kamper](https://www.kamperh.com/)
*Equal contribution
## Quickstart
We use `torch.hub` to make loading the model easy -- no cloning of the repo needed. The steps to perform inference are simple:
1. **Install dependancies**: we have 3 inference dependencies only `torch`, `torchaudio`, and `numpy`. Python must be at version 3.10 or greater, and torch must be v2.0 or greater.
2. **Load models**: load the WavLM encoder and HiFiGAN vocoder:
```python
import torch, torchaudio
knn_vc = torch.hub.load('bshall/knn-vc', 'knn_vc', prematched=True, trust_repo=True, pretrained=True)
# Or, if you would like the vocoder trained not using prematched data, set prematched=False.
```
3. **Compute features** for input and reference audio:
```python
src_wav_path = '<path to arbitrary 16kHz waveform>.wav'
ref_wav_paths = ['<path to arbitrary 16kHz waveform from target speaker>.wav', '<path to 2nd utterance from target speaker>.wav', ...]
query_seq = knn_vc.get_features(src_wav_path)
matching_set = knn_vc.get_matching_set(ref_wav_paths)
```
4. **Perform the kNN matching and vocoding**:
```python
out_wav = knn_vc.match(query_seq, matching_set, topk=4)
# out_wav is (T,) tensor converted 16kHz output wav using k=4 for kNN.
```
That's it! These default settings provide pretty good results, but feel free to modify the kNN `topk` or use the non-prematched vocoder.
Note: the target speaker from `ref_wav_paths` _can be anything_, but should be clean speech from the desired speaker. The longer the cumulative duration of all reference waveforms, the better the quality will be (but the slower it will take to run). The improvement in quality diminishes beyond 5 minutes of reference speech.
## Checkpoints
Under the releases tab of this repo we provide three checkpoints:
- The frozen WavLM encoder taken from the [original WavLM authors](https://github.com/microsoft/unilm/tree/master/wavlm), which we host here for convenience and torch hub integration.
- The HiFiGAN vocoder trained on layer 6 of WavLM features.
- The HiFiGAN vocoder trained on **prematched** layer 6 of WavLM features (the best model in the paper).
For the HiFiGAN models we provide both the generator inference checkpoint and full training checkpoint with optimizer states.
The performance on the LibriSpeech dev-clean set is summarized:
| checkpoint | WER (%) | CER (%) | EER (%) |
| ----------- | :-----------: | :----: | :--: |
| [kNN-VC with prematched HiFiGAN](https://github.com/bshall/knn-vc/releases/download/v0.1/prematch_g_02500000.pt) | 6.29 | 2.34 | 35.73 |
| [kNN-VC wExcerpt of 8,593 characters
Read on GitHub
19
3
Emmanuel Schmidbauer
1
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:26eb1d8b89aa9f5e, topic:pytorch
matched fp:26eb1d8b89aa9f5e, topic:speech-synthesis