Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
MARS5 speech model (TTS) from CAMB.AI
| Date | Stars |
|---|---|
| 2026-07-24 | 2816 |
| 2026-07-25 | 2816 |
| 2026-07-28 | 2816 |
| 2026-07-30 | 2818 |
| 2026-08-06 | 2818 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# MARS5: A novel speech model for insane prosody
<div id="top" align="center">
<h3>
<a href="https://www.loom.com/share/a6e7c6658f9f4b09a696926a98dd6fcc"> Why MARS5? </a> |
<a href="https://github.com/Camb-ai/MARS5-TTS/blob/master/docs/architecture.md"> Model Architecture </a> |
<a href="https://6b1a3a8e53ae.ngrok.app/"> Samples </a> |
<a href="https://camb.ai/"> Camb AI Website </a></h3>
[](https://github.com/Camb-ai/MARS5-TTS/stargazers)
[](https://discord.gg/FFQNCSKSXX)
[](https://huggingface.co/CAMB-AI/MARS5-TTS)
[](https://colab.research.google.com/github/Camb-ai/mars5-tts/blob/master/mars5_demo.ipynb)
</div>
# Updates
<> July 5, 2024: Latest AR checkpoint released: higher stability of output. Very big update coming soon!
# Approach
This is the repo for the MARS5 English speech model (TTS) from CAMB.AI.
The model follows a two-stage AR-NAR pipeline with a distinctively novel NAR component (see more info in the [Architecture](docs/architecture.md)).
With just 5 seconds of audio and a snippet of text, MARS5 can generate speech even for prosodically hard and diverse scenarios like sports commentary, anime and more. Check out our demo:
https://github.com/Camb-ai/MARS5-TTS/assets/23717819/3e191508-e03c-4ff9-9b02-d73ae0ebefdd
Watch full video here: [](https://www.youtube.com/watch?v=bmJSLPYrKtE)
**Figure**: The high-level architecture flow of MARS5. Given text and a reference audio, coarse (L0) encodec speech features are obtained through an autoregressive transformer model. Then, the text, reference, and coarse features are refined in a multinomial DDPM model to produce the remaining encodec codebook values. The output of the DDPM is then vocoded to produce the final audio.
Because the model is trained on raw audio together with byte-pair-encoded text, it can be steered with things like punctuation and capitalization.
E.g. To add a pause, add a comma to that part in the transcript. Or, to emphasize a word, put it in capital letters in the transcript.
This enables a fairly natural way for guiding the prosody of the generated output.
Speaker identity is specified using an audio reference file between 2-12 seconds, with lengths around 6s giving optimal results.
Further, by providing the transcript of the reference, MARS5 enables one to do a '_deep clone_' which improves the quality of the cloning and output, at the cost of taking a bit longer to produce the audio.
For more details on this and other performance and model details, please see the [docs folder](docs/architecture.md).
## Quick links
- [CAMB.AI website](https://camb.ai/) (access MARS in 140+ languages for TTS and dubbing)
- Technical details and architecture: [in the docs folder](docs/architecture.md)
- Colab quickstart: [](https://colab.research.google.com/github/Camb-ai/mars5-tts/blob/master/mars5_demo.ipynb)
- Sample page with a few hard prosodic samples: [https://camb-ai.github.io/MARS5-TTS/](https://camb-ai.github.io/MARS5-TTS/)
- Online demo: [here](https://6b1a3a8e53ae.ngrok.app/)
## Quickstart
We use `torch.hub` to make loading the model easy -- no cloning of the repo needed. The steps to perform inference are simple:
1. **Installation using pip**:
Requirements:
- Python >= 3.10
- Torch >= 2.0
- Torchaudio
- Librosa
- Vocos
- Encodec
- safetensors
- regex
```bash
pip install --upgrade torch torchaudio librosa voExcerpt of 13,237 characters
Read on GitHub
24
16
9
Arnav Mehta
7
NourAlMerey · @cambai
3
Nihaal Nz
3
2
1
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:296fda6c1e7a9d2b, topic:text-to-speech, topic:voice-cloning, topic:speech-synthesis