Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
VibeVoiceFusion is a full-stack, multi-speaker voice generation web system featuring LoRA fine-tuning, batch generation, and VRAM optimization. Based on Microsoft's VibeVoice (AR + diffusion architecture)
| Date | Stars |
|---|---|
| 2026-07-24 | 483 |
| 2026-07-25 | 483 |
| 2026-07-28 | 483 |
| 2026-07-30 | 483 |
| 2026-07-31 | 486 |
| 2026-08-06 | 486 |
Today
— stars today
This week
+3 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.62%/day
# VibeVoiceFusion
<div align="center">
<img src="frontend/public/icon-rect-pulse.svg" alt="VibeVoiceFusion Logo" width="120"/>
**A Complete Web Application for Multi-Speaker Voice Generation**
*Built on Microsoft's VibeVoice Model*
[](LICENSE)
[](https://www.python.org/)
[](https://www.typescriptlang.org/)
[](Dockerfile)
[](https://hub.docker.com/r/zhaokundev/vibevoicefusion)
[](https://hub.docker.com/r/zhaokundev/vibevoicefusion)
[](https://hub.docker.com/r/zhaokundev/vibevoicefusion)
[English](README.md) | [简体中文](README_zh.md)
[Features](#features) • [Demo Samples](#demo-samples) • [Get Started](#get-started) • [Documentation](#documentation) • [Community](#community) • [Contributing](#contributing)
</div>
---
## Overview
### Purpose
VibeVoiceFusion is a **web application** for generating high-quality, multi-speaker synthetic speech with voice cloning capabilities. Built on Microsoft's VibeVoice model (AR + diffusion architecture), this project provides a complete full-stack solution with voice generation, LoRA fine-tuning, dataset management, batch generation, and advanced VRAM optimization features.
**Key Goals:**
- Provide a user-friendly interface for voice generation without requiring coding knowledge
- Enable efficient multi-speaker dialog synthesis with distinct voice characteristics
- Support LoRA fine-tuning for custom voice adaptation and style transfer
- Generate multiple audio variations in batch with different random seeds
- Optimize memory usage for consumer-grade GPUs (10GB+ VRAM)
- Support bilingual workflows (English/Chinese)
- Offer both web UI and CLI interfaces for different use cases
<div align="center">
<a href="https://youtu.be/J9pmcOBWN4c" target="_blank">
<img src="docs/images/VibevoiceFusion.png" alt="Video Introduction" width="700"/>
</a>
</div>
### Principle
VibeVoice combines **autoregressive (AR)** and **diffusion** techniques for text-to-speech synthesis:
1. **Text Processing**: Input text is tokenized and processed through a Qwen-based language model backbone
2. **Voice Encoding**: Reference voice samples are encoded into acoustic and semantic embeddings
3. **AR Generation**: The model autoregressively generates speech tokens conditioned on text and voice embeddings
4. **Diffusion Refinement**: A DPM-Solver-based diffusion head converts tokens to high-quality audio waveforms
5. **Voice Cloning**: The unified processor preserves speaker characteristics from reference audio samples
**Technical Highlights:**
- **Model Architecture**: Qwen backbone + VAE acoustic tokenizer + semantic encoder + diffusion head
- **Quantization**: Float8 (FP8 E4M3FN) support for ~50% VRAM reduction with minimal quality loss
- **Layer Offloading**: Dynamic CPU/GPU memory management for running on limited VRAM
- **Attention Mechanism**: PyTorch native SDPA for maximum compatibility
### Features
#### Quick Generation
- **One-Click Generation**: Generate voice without creating projects, speakers, or sessions
- **Voice Source Options**:
- Upload custom audio files (WAV, MP3, M4A, FLAC, WebM) - up to 4 files
- Select from preset voice samples with language/gender filters
- **Auto Mode Detection**: Automatically detects dialogue vs narration format
- **Multi-Voice Support**: Use up to 4 voice prompts for generation
- **Generation History**: Persistent history with expandable details, bulk delete
- **Per-Item Progress**: Real-time progreExcerpt of 27,135 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:3fd4899a9fe9b01e, topic:fine-tuning, topic:lora, desc:fine-tuning
matched fp:3fd4899a9fe9b01e, topic:tts, topic:speech-synthesis, readme:text-to-speech