Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
dInfer: An Efficient Inference Framework for Diffusion Language Models
| Date | Stars |
|---|---|
| 2026-07-31 | 476 |
| 2026-08-06 | 477 |
Today
+1 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center"> <img src="assets/logo.svg" width="40%" alt="dInfer" /> </div> <h4 align="center"> [](./LICENSE) [](https://huggingface.co/inclusionAI/LLaDA-MoE-7B-A1B-Instruct) [](https://arxiv.org/abs/2510.08666) <!-- [![arXiv][arxiv-image]][arxiv-url] --> </h4> ## Introduction dInfer is an efficient and extensible inference framework for dLLMs. As illustrated in the following architecture, it modularizes inference into four components: *model*, *diffusion iteration manager*, *decoder* and *KV-cache manager*. It provides well-designed APIs for flexible algorithms combinations in each component. It now supports batched inference for improved throughput. <p align="center"> <img src="assets/Framework2.png" alt="dInfer v0.1 architecture" width="600"> <br> <b>Figure</b>: Overall Architecture of dInfer </p> dInfer supports multiple dLLM variants, including LLaDA, LLaDA-MoE and LLaDA2. ## News **\[2025/12/21\]** release v0.2. The major features of this release can be found [here](https://github.com/inclusionAI/dInfer/releases/tag/v0.2.0). **\[2025/12/10\]** Support and speed up the formal version of block diffusion LLMs (LLaDA2-mini and LLaDA2-flash). Support quant versions of LLaDA2-mini and LLaDA2-flash. **\[2025/11/15\]** Support the inference on block diffusion LLMs (LLaDA2-mini-preview and LLaDA2-flash-preview). **\[2025/10/10\]** Release the first version of the dInfer framework. ## Contents - [Supported Models](#supported-models) - [Quick Start](#quick-start) - [Benchmark Results](#benchmark-results) ## Supported Models dInfer supports multiple diffusion language model variants with different architectures and sizes. Below are the HuggingFace model links and their corresponding implementation files: | Model | Size | Implementation | HuggingFace Link | |-------|------|----------------|------------------| | LLaDA2.0-mini | 16B | [LLaDA2MoeModelLM](python/dinfer/model/modeling_llada2_moe.py) | [inclusionAI/LLaDA2.0-mini](https://huggingface.co/inclusionAI/LLaDA2.0-mini) | | LLaDA2.0-flash | 100B | [LLaDA2MoeModelLM](python/dinfer/model/modeling_llada2_moe.py) | [inclusionAI/LLaDA2.0-flash](https://huggingface.co/inclusionAI/LLaDA2.0-flash) | | LLaDA2.0-mini-preview | 16B | [LLaDA2MoeModelLM](python/dinfer/model/modeling_llada2_moe.py) | [inclusionAI/LLaDA2.0-mini-preview](https://huggingface.co/inclusionAI/LLaDA2.0-mini-preview) | | LLaDA2.0-flash-preview | 100B | [LLaDA2MoeModelLM](python/dinfer/model/modeling_llada2_moe.py) | [inclusionAI/LLaDA2.0-flash-preview](https://huggingface.co/inclusionAI/LLaDA2.0-flash-preview) | | LLaDA-MoE-7B-A1B-Base | 7B | [LLaDAMoeModelLM](python/dinfer/model/modeling_fused_olmoe.py) | [inclusionAI/LLaDA-MoE-7B-A1B-Base](https://huggingface.co/inclusionAI/LLaDA-MoE-7B-A1B-Base) | | LLaDA-MoE-7B-A1B-Instruct | 7B | [LLaDAMoeModelLM](python/dinfer/model/modeling_fused_olmoe.py) | [inclusionAI/LLaDA-MoE-7B-A1B-Instruct](https://huggingface.co/inclusionAI/LLaDA-MoE-7B-A1B-Instruct) | | LLaDA-8B-Base | 8B | [LLaDAModelLM](python/dinfer/model/modeling_llada.py) | [GSAI-ML/LLaDA-8B-Base](https://huggingface.co/GSAI-ML/LLaDA-8B-Base) | | LLaDA-8B-Instruct | 8B | [LLaDAModelLM](python/dinfer/model/modeling_llada.py) | [GSAI-ML/LLaDA-8B-Instruct](https://huggingface.co/GSAI-ML/LLaDA-8B-Instruct) | | LLaDA-1.5 | 8B | [LLaDAModelLM](python/dinfer/model/modeling_llada.py) | [GSAI-ML/LLaDA-1.5](https://huggingface.co/GSAI-ML/LLaDA-1.5) | ## Quick Start ### Install dInfer ``` git clone https://github.com/inclusionAI/dInfer.git cd dInfer pip install . ``` To use it with vLLM backend (it works with LLaDA and LLaDA-MoE), please install vLLM. ``` pip install vllm==0.10.2 ``` To use it with SGLang backend (it works with LLaDA2), please install SGLan
Excerpt of 9,375 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:6992507553454cca, llm:Repository description: 'dInfer: An Efficient Inference Framework for Diffusion Language Models' (Python project).
matched fp:6992507553454cca, llm:Repository description: 'dInfer: An Efficient Inference Framework for Diffusion Language Models' (Python project).
matched fp:6992507553454cca, llm:Repository description: 'dInfer: An Efficient Inference Framework for Diffusion Language Models' (Python project).