Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Multilingual TTS model with voice cloning and duration control, based on T5Gemma encoder-decoder LLM
| Date | Stars |
|---|---|
| 2026-07-24 | 311 |
| 2026-07-25 | 311 |
| 2026-07-28 | 311 |
| 2026-07-30 | 310 |
| 2026-08-06 | 310 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# T5Gemma-TTS
[](https://arxiv.org/abs/2604.01760)
[](https://huggingface.co/Aratako/T5Gemma-TTS-2b-2b)
[](https://huggingface.co/spaces/Aratako/T5Gemma-TTS-Demo)
[](LICENSE)
**[日本語版 README はこちら](README_ja.md)**
Training and inference code for **T5Gemma-TTS**, a multilingual Text-to-Speech model based on the Encoder-Decoder LLM architecture. This repository provides scripts for data preprocessing, training (including LoRA fine-tuning), and inference.
For model details, audio samples, and technical information, please refer to the [model card](https://huggingface.co/Aratako/T5Gemma-TTS-2b-2b).
## Updates
**2026/04/03**
- 🔥 **Technical report is out!** Read the paper on arXiv: [https://arxiv.org/abs/2604.01760](https://arxiv.org/abs/2604.01760)
**2025/12/17**
- Added batch inference support for generating multiple audio variations in parallel from the same input (Gradio UI)
- Added quantized model support (8-bit/4-bit encoder quantization) for reduced VRAM usage
- Added `--low_vram` option (CPU offloading for XCodec2/Whisper) for memory-constrained environments
- Added Apple Silicon (MPS) support
- Added PyTorch 2.9+ compatibility
## Features
- **Multilingual TTS**: Supports English, Chinese, and Japanese
- **Voice Cloning**: Zero-shot voice cloning from reference audio
- **Duration Control**: Explicit control over generated audio length (auto-estimation when not specified)
- **Batch Generation**: Generate multiple audio variations in parallel from the same input (Gradio UI)
- **Flexible Training**: Full training, fine-tuning, and LoRA fine-tuning support
- **Multiple Inference Options**: Command-line, HuggingFace format, and Gradio UI
## Installation
```bash
git clone https://github.com/Aratako/T5Gemma-TTS.git
cd T5Gemma-TTS
pip install -r requirements.txt
```
**Note**: For GPU support, install PyTorch with CUDA before running `pip install`:
```bash
pip install "torch<=2.8.0" torchaudio --index-url https://download.pytorch.org/whl/cu128
```
## Known Issues
- **Windows**: On some native Windows environments, inference may exhibit unstable behavior such as inconsistent generation times or occasional hangs. This issue has been observed in my testing but the root cause is still under investigation. If you experience similar problems, consider using WSL2 or Docker as a workaround.
## Tested Environments
The following environments have been tested by the developer. Other configurations may work but are not guaranteed.
| Platform | Status | Notes |
|----------|--------|-------|
| Linux + CUDA | ✅ Tested | |
| Windows + CUDA (Docker) | ✅ Tested | Native Windows has known issues |
| Apple Silicon (MPS) | ✅ Tested | M4 Max MacBook Pro |
## Quick Start
### Basic Inference (HuggingFace Format)
```bash
python inference_commandline_hf.py \
--model_dir Aratako/T5Gemma-TTS-2b-2b \
--target_text "Hello, this is a test of the text to speech system."
```
### Voice Cloning
```bash
python inference_commandline_hf.py \
--model_dir Aratako/T5Gemma-TTS-2b-2b \
--target_text "Hello, this is a test of the text to speech system." \
--reference_text "This is a reference." \
--reference_speech path/to/reference.wav
```
### Duration Control
```bash
# Specify target duration in seconds
python inference_commandline_hf.py \
--model_dir Aratako/T5Gemma-TTS-2b-2b \
--target_text "Hello, this is a test of the text to speech system." \
--target_duration 5.0
```
**Note**: If `--target_duration` is not specified, the system automatically calculates an appropriate duration based on phoneme count and language-specific pacing rules. This calculation is approximate, so if the result isn't as expected, tExcerpt of 13,520 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:349613e7cc4a9fe3, topic:text-to-speech, topic:tts, topic:voice-cloning