Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A Framework for Speech, Language, Audio, Music Processing with Large Language Model
| Date | Stars |
|---|---|
| 2026-07-24 | 1049 |
| 2026-07-25 | 1049 |
| 2026-07-28 | 1049 |
| 2026-07-30 | 1049 |
| 2026-07-31 | 1051 |
| 2026-08-06 | 1051 |
Today
— stars today
This week
+2 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.19%/day
<div align="center">
<h1>
SLAM-LLM
</h1>
<p>
<b>SLAM-LLM</b> is a deep learning toolkit that allows researchers and
developers to train custom multimodal large language model (MLLM), focusing on <b>S</b>peech, <b>L</b>anguage, <b>A</b>udio, <b>M</b>usic processing. We provide detailed recipes for training and high-performance checkpoints for inference. <br>
</p>
<p>
<img src="docs/logo.jpg" alt="SLAM-LLM Logo" style="width: 200px; height: 200px;">
</p>
<p>
</p>
<a href="https://github.com/ddlBoJack/SLAM-LLM"><img src="https://img.shields.io/badge/Platform-linux-lightgrey" alt="version"></a>
<a href="https://github.com/ddlBoJack/SLAM-LLM"><img src="https://img.shields.io/badge/Cuda-11.8+-orange" alt="version"></a>
<a href="https://github.com/ddlBoJack/SLAM-LLM"><img src="https://img.shields.io/badge/PyTorch-2.01+-brightgreen" alt="python"></a>
<a href="https://github.com/ddlBoJack/SLAM-LLM"><img src="https://img.shields.io/badge/License-MIT-red.svg" alt="mit"></a>
</div>
# Table of Contents
1. [News](#news)
2. [Installation](#installation)
3. [Usage](#usage)
- [List of Recipes](#list-of-recipes)
- [Configuration Priority](#configuration-priority)
4. [Features](#features)
5. [Acknowledge](#acknowledge)
6. [Citation](#citation)
# News
- [Update Jan. 15, 2026] The [SLAM-LLM Framework](https://arxiv.org/abs/2601.09385v1) was accepted by IEEE Journal of Selected Topics in Signal Processing (JSTSP), where we put the framework design and some unpublished experiments and insights for speech, audio, and music MLLM.
- [Update Apr. 24, 2025] We have supported [large-scale industrial training](examples/aispeech_asr/README.md), suitable for datasets on the order of 100,000 hours. Its main features include:
- **Support for multi-task training:** Designed to support tasks such as ASR and ST through a unified data format.
- **Dynamic prompt selection:** Supports random selection from multiple prompts.
- **Iterative dataset:** Uses an iterative dataset format to reduce startup time for large datasets.
- **Deepspeed training:** Supports DeepSpeed training to significantly reduce memory usage.
- **Multi-machine multi-GPU inference:** Supports distributed inference across multiple machines and GPUs to reduce evaluation time.
- **Dynamic frame batching:** Dynamically combines frames based on audio size rather than using a fixed batch size, significantly reducing training and evaluation time (reduces training time by 3/4 for 100,000 hours of data).
- [Update Apr. 24, 2025] We have supported the Deepspeed, checkout the instruction #Fine-tuning using Deepspeed at [here](examples/asr_librispeech/README.md).
- [Update Jan. 22, 2025] 🔥🔥🔥 Full reproduction (including all data preparation, model training, and inference) for [SLAM-Omni](examples/s2s/README.md) has been supported.
- SLAM-Omni is a **timbre-controllable** voice interaction system that requires only **single-stage training** and minimal resources to achieve high-quality, end-to-end speech dialogue, supporting multi-turn conversations in both Chinese and English. ([paper](https://arxiv.org/abs/2412.15649), [demo](https://slam-omni.github.io))
- We have fully reproduced the **training and inference** processes of SLAM-Omni and open-sourced all related training datasets. The provided code framework theoretically supports all codec-based spoken dialogue models. Additionally, we offer the reproduction code for [Mini-Omni](https://github.com/gpt-omni/mini-omni).
<table class="center">
<tr>
<td width=50% style="border: none">
<video controls autoplay loop src="https://github.com/user-attachments/assets/73597edb-0d66-453b-b10c-8cf8dd3cae18" muted="false"></video>
</td>
<td width=50% style="border: none">
<video controls autoplay loop src="https://github.com/user-attachments/assets/7a797491-0509-4da8-8662-f2107bd8856a" muted="false"></video>
</td>
</Excerpt of 12,275 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:ce2c7a48551e599d, topic:peft, readme:fine-tuning, readme:fine tuning
matched fp:ce2c7a48551e599d, topic:audio-processing
matched fp:ce2c7a48551e599d, name:slam, readme:slam
