Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
| Date | Stars |
|---|---|
| 2026-07-31 | 671 |
| 2026-08-06 | 699 |
| 2026-08-14 | 728 |
| 2026-08-18 | 739 |
| 2026-08-19 | 747 |
| 2026-08-20 | 749 |
| 2026-08-21 | 753 |
| 2026-08-22 | 754 |
| 2026-08-24 | 759 |
| 2026-08-25 | 762 |
| 2026-08-26 | 765 |
| 2026-08-27 | 767 |
| 2026-08-28 | 776 |
| 2026-08-29 | 780 |
| 2026-08-30 | 782 |
| 2026-08-31 | 785 |
| 2026-09-01 | 788 |
| 2026-09-02 | 795 |
| 2026-09-03 | 799 |
| 2026-09-04 | 806 |
| 2026-09-05 | 807 |
| 2026-09-06 | 807 |
| 2026-09-07 | 812 |
| 2026-09-08 | 816 |
| 2026-09-09 | 822 |
| 2026-09-10 | 822 |
| 2026-09-11 | 825 |
| 2026-09-12 | 826 |
| 2026-09-13 | 827 |
| 2026-09-14 | 830 |
| 2026-09-15 | 833 |
| 2026-09-16 | 835 |
| 2026-09-17 | 837 |
| 2026-09-18 | 841 |
| 2026-09-19 | 843 |
| 2026-09-20 | 844 |
Today
+1 stars today
This week
+17 stars this week
This month
+91 stars this month
Momentum
56.0
growth rate 2.06%/day
<div align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/speculators/main/docs/assets/branding/speculators-logo-white.svg" />
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/vllm-project/speculators/main/docs/assets/branding/speculators-logo-black.svg" />
<img alt="Speculators logo" src="https://raw.githubusercontent.com/vllm-project/speculators/main/docs/assets/branding/speculators-logo-black.svg" height="64" />
</picture>
[](https://github.com/vllm-project/speculators/blob/main/LICENSE) [](https://pypi.org/project/speculators/) [](https://docs.vllm.ai/projects/speculators/en/latest/) [](https://pypi.org/project/speculators/) [](https://buildkite.com/vllm/speculators)
</div>
## Overview
Speculators is a library for training speculative decoding draft models that deploy directly to LLM inference engines like vLLM. Speculative decoding is a lossless technique that speeds up LLM inference by using a smaller, faster draft model (i.e. "the speculator") to propose tokens, which are then verified by the larger base model, reducing latency without compromising output quality. The speculator intelligently drafts multiple tokens ahead of time, and the base model verifies them in a single forward pass. This approach boosts performance without sacrificing output quality, as every accepted token is guaranteed to match what the main model would have generated on its own.
Speculators standardizes this process by providing a productionized end-to-end framework to train draft models with reusable formats and tools. Trained models can seamlessly run in vLLM, enabling the deployment of speculative decoding in production-grade inference servers.
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/speculators/main/docs/assets/branding/speculators-user-flow-dark.svg" />
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/vllm-project/speculators/main/docs/assets/branding/speculators-user-flow-light.svg" />
<img alt="Speculators user flow diagram" src="https://raw.githubusercontent.com/vllm-project/speculators/main/docs/assets/branding/speculators-user-flow-light.svg" />
</picture>
</p>
______________________________________________________________________
💬 Join us on the [vLLM Community Slack](https://inviter.co/vllm-slack) and share your questions, thoughts, or ideas in:
- `#speculators`
- `#feat-spec-decode`
🎥 Watch our Office Hours presentation: [Video](https://www.youtube.com/live/2ISAr_JVGLs) | [Slides](https://docs.google.com/presentation/d/1s4eAb7v-rdZt8smyULBJWGXjJXrgFTZWnwqYa2-h1l4/edit?slide=id.g3365e070742_6_0#slide=id.g3365e070742_6_0)
______________________________________________________________________
## 🚀 What's New!
Big updates have landed in Speculators! To get a more in-depth look, check out the [Speculators documentation](https://docs.vllm.ai/projects/speculators/en/latest/).
Some of the exciting new features include:
- **Multi-Node Online Training via hs_connectors**: Added the [`hs_connectors`](https://github.com/vllm-project/speculators/tree/main/hs_connectors) plugin package with pluggable backends for transferring hidden states between vLLM and the trainer across nodes. The file-based backend uses a shared filesystem, while the Mooncake backend leverages a distributed store for environments without shared storage, enabling online speculator training at multi-node scale.
- **DSpark TrainingExcerpt of 13,759 characters
Read on GitHub
100
72
65
46
38
25
23
Guan-Ming Chiu · National Taiwan University · Taiwan
18
Eldar Kurtić
17
Mark Kurtz · Red Hat · United States
13
13
Neural Magic · United States
9
4
4
4
3
3
Aidan Reilly · Red Hat · Ireland
3
SwekeR
2
Michael Goin · @vllm-project @redhatofficial · United States
2
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:493bdbe3e7ea1ef1, desc:speculative decoding