Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A fast multimodal LLM for real-time voice
| Date | Stars |
|---|---|
| 2026-07-31 | 4503 |
| 2026-08-02 | 4503 |
| 2026-08-05 | 4507 |
| 2026-08-06 | 4507 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center">
<picture>
<img alt="Ultravox" src="https://go.ultravox.ai/rs/577-CTA-193/images/uv-logo-600x140.png">
</picture>
</p>
<h3 align="center">
A fast multimodal LLM designed for real-time voice interactions
</h3>
_Latest News_
* 2025/12 - [Ultravox 0.7](https://huggingface.co/collections/fixie-ai/ultravox-v07) available
* 2025/06 — [Ultravox 0.6](https://huggingface.co/collections/fixie-ai/ultravox-v06-6865e7885cf904e21c31da03) available
* 2025/02 — [Ultravox 0.5](https://github.com/fixie-ai/ultravox/releases/tag/v0.5) available
* 2024/11 — [Ultravox 0.4.1](https://github.com/fixie-ai/ultravox/releases/tag/v0.4.1) available
* 2024/08 — [Ultravox 0.4](https://github.com/fixie-ai/ultravox/releases/tag/v0.4) available
* 2024/08 — [Ultravox 0.3](https://github.com/fixie-ai/ultravox/releases/tag/v0.3) available
* 2024/08 — Preview of Ultravox APIs available, more information [here](https://fixie-ai.github.io/ultradox/)
_Key Links_
* [Ultravox Realtime](https://ultravox.ai) — Build real-time Voice AI agents on top of the Ultravox model
* [Hugging Face](https://huggingface.co/fixie-ai) — Our Hugging Face page
---
# About
Ultravox is a new kind of multimodal LLM that can understand text as well as human speech, without the need for a separate Audio Speech Recognition (ASR) stage. Building on research like [AudioLM](https://arxiv.org/abs/2209.03143), [SeamlessM4T](https://ai.meta.com/blog/seamless-m4t/), [Gazelle](https://tincans.ai/slm), [SpeechGPT](https://github.com/0nutation/SpeechGPT/tree/main/speechgpt), and others, Ultravox is able to extend any open-weight LLM with a multimodal projector that converts audio directly into the high-dimensional space used by LLM. We've trained versions on Llama 3, Mistral, and Gemma. This direct coupling allows Ultravox to respond much more quickly than systems that combine separate ASR and LLM components. In the future this will also allow Ultravox to natively understand the paralinguistic cues of timing and emotion that are omnipresent in human speech.
Ultravox currently takes in audio and emits streaming text. As we evolve the model, we'll train it to be able to emit a stream of speech tokens that can then be converted directly into raw audio by an appropriate unit vocoder.
Our default model is built on top of Llama 3.3 70B. We also have an 8B variant available on Hugging Face.
Ultravox can be trained against any open-weight model. See below for more details on training.
### Demo
See Ultravox in action on our [demo page](https://demo.ultravox.ai). You can build your own voice-to-voice agents on our Realtime platform at ultravox.ai.
### Discord
Join us on our Discord server [here](https://discord.gg/Qw6KHxv8YB).
### Jobs
If you're interested in working on Ultravox fulltime, we're hiring! Check out our jobs page [here](https://careers.fixie.ai).
### Inference Server
You can try out Ultravox using your own audio content (as a WAV file) by spinning up an Ultravox instance on our partner, BaseTen: [https://www.baseten.co/library/ultravox/](https://www.baseten.co/library/ultravox/). They offer free credits to get started.
If you're interested in running Ultravox in a real-time capacity, we offer a set of managed APIs as well. You can learn more about getting access to those [here](https://docs.ultravox.ai).
### Model
You can download the latest weights from the [Ultravox Hugging Face page](https://huggingface.co/fixie-ai/).
### Architecture
[](https://docs.google.com/presentation/d/1ey81xuuMzrJaBwztb_Rq24Cit37GQokD2aAes_KkGVI/edit)
# Contributing
Read on if you're interested in training your own version of Ultravox.
## Environment Setup (Mac)
Install the basic tools:
- [`Homebrew`](https://brew.sh) is a package manager for MacOS that also mostly works for Linux. If you're running Debian or Ubuntu Linux, you can alternativelyExcerpt of 9,691 characters
Read on GitHub
44
39
12
11
10
2
1
1
Matthew So · Okaya · United States
1
Ikko Eltociear Ashimine · Japan
1
1
Ofek Lev · @datadog
1
Freddy Boulton
1
Simon Willison · Datasette · United States
1
Walter Korman · @vercel · United States
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:d2720fbda9e3210a, topic:llm