Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
| Date | Stars |
|---|---|
| 2026-07-24 | 1019 |
| 2026-07-25 | 1019 |
| 2026-07-28 | 1019 |
| 2026-07-30 | 1019 |
| 2026-08-06 | 1019 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# SpargeAttention
## Recommended API
We highly recommend using the `spas_sage2_attn_meansim_topk_cuda` and `block_sparse_sage2_attn_cuda` APIs. They are plug-and-play and customizable:
### Plug-and-Play API
```python
from spas_sage_attn import spas_sage2_attn_meansim_topk_cuda
attn_output = spas_sage2_attn_meansim_topk_cuda(q, k, v, topk=0.5, is_causal=False)
```
You can adjust `topk` to balance between attention accuracy (higher `topk` is more accurate) and sparsity (lower `topk` is more sparse).
### Customize your Block-Sparse Mask
```python
from spas_sage_attn import block_sparse_sage2_attn_cuda
attn_output = block_sparse_sage2_attn_cuda(q, k, v, mask_id=None):
```
In this API, we **support computing attention in any block sparse mask per attention head**. Specifically, the per-head attention mask `mask_id` has shape `(batch_size, num_heads, ⌈seq_len / 128⌉, ⌈seq_len // 64⌉)` and consists of `0` and `1`. Currently, the block size is 128×64.
---
The official implementation of [SpargeAttn](https://arxiv.org/abs/2502.18137), a universal training-free sparse attention accelerating language, image, and video models.
<div align="center"> <h2>SpargeAttention: Accurate and Training-free Sparse Attention<br>Accelerating Any Model Inference</h2> <a href="https://huggingface.co/papers/2502.18137"> <img src="https://img.shields.io/static/v1?label=Daily%20papers&message=HuggingFace&color=yellow" alt="Daily papers: HuggingFace"> </a> <a href="https://arxiv.org/abs/2502.18137"> <img src="https://img.shields.io/badge/arXiv-2502.18137-b31b1b.svg" alt="arXiv:2502.18137"> </a> </div>
<div align="center">
<a href="https://jt-zhang.github.io/" target="_blank">Jintao Zhang</a><sup></sup> |
<a href="https://xiang-cd.github.io/cv" target="_blank">Chendong Xiang</a><sup></sup> |
<a href="https://github.com/jason-huang03" target="_blank">Haofeng Huang</a><sup></sup> |
<a href="https://haochengxi.github.io/" target="_blank">Haocheng Xi</a><sup></sup> |
<a href="" target="_blank">Jia Wei</a><sup></sup> |
<a href="https://ml.cs.tsinghua.edu.cn/~jun/index.shtml" target="_blank">Jun Zhu</a><sup></sup> |
<a href="https://ml.cs.tsinghua.edu.cn/~jianfei" target="_blank">Jianfei Chen</a><sup></sup>
</div>
<!-- Jintao Zhang, Chendong Xiang, Haofeng Huang, Haocheng Xi, Jia Wei, Jun Zhu, Jianfei Chen -->
<br>
<p align="center">
<img src="./assets/speed_comparison.png" width="81%" alt="speed comparison.">
</p>
<p align="center">
<img src="./assets/overview.png" width="90%" alt="overview.">
</p>
## Project Updates
- **Please use the `spas_sage2_attn_meansim_topk_cuda` and `block_sparse_sage2_attn_cuda` APIs.**
- [2025-07]: Release a Triton Kernel example.
- [2025-06]: SpargeAttn based on [SageAttention2++](https://arxiv.org/abs/2505.21136) is released.
- [2025-05]: Add a **very simple usage without tuning or calibration**: `o = spas_sage2_attn_meansim_topk_cuda(q, k, v)`.
- [2025-05]: 🎉SpargeAttn and [SageAttention2](https://github.com/thu-ml/SageAttention) are accepted by ICML 2025!
- [2025-03] Support high acceleration on more GPUs, e.g., H100.
## Installation
### Base environment
+ `python>=3.9` , `torch>=2.3.0`
- `CUDA`:
+ `>=12.8` for Blackwell, `>=12.4` for fp8 support on Ada, `>=12.3` for fp8 support on Hopper, `>=12.0` for Ampere
### Install Package
```bash
pip install ninja # for parallel compilation
python setup.py install # or pip install -e .
```
## Available API
- `spas_sage2_attn_meansim_topk_cuda`: SpargeAttn based on [SageAttention2](https://github.com/thu-ml/SageAttention) that **we recommend using.**
- `spas_sage2_attn_meansim_cuda`: SpargeAttn based on [SageAttention2](https://github.com/thu-ml/SageAttention) that we do not recommend.
- `spas_sage_attn_meansim_topk_cuda`: SpargeAttn based on [SageAttention](https://github.com/thu-ml/SageAttention) that **we recommend using.**
- `spas_sage_attn_meansim_cuda`: SpargeAttn based on [SageAttention](https://github.com/thu-ml/SageAtteExcerpt of 9,293 characters
Read on GitHub
Jintao Zhang · @thu-ml, Tsinghua University · China
60
21
Haoxu Wang · Tsinghua University · China
10
Haofeng Huang · Tsinghua University · China
7
7
1
1
1
1
Sepehr Sameni · Switzerland
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:7afcb4cceab114b3, topic:quantization, readme:fp8, readme:sparsity
matched fp:7afcb4cceab114b3, topic:llm
matched fp:7afcb4cceab114b3, topic:video-generation