Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
| Date | Stars |
|---|---|
| 2026-07-24 | 3502 |
| 2026-07-25 | 3502 |
| 2026-07-28 | 3518 |
| 2026-07-30 | 3518 |
| 2026-08-06 | 3518 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# SageAttention <!-- We are continuously updating more features. You could **Star** and **Watch** our repository to stay updated. --- --> This repository provides the official implementation of SageAttention, SageAttention2, and SageAttention2++, which achieve surprising speedup on most GPUs without lossing accuracy across all models in a plug-and-play way. **SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration** Jintao Zhang, Jia Wei, Haofeng Huang, Pengle Zhang, Jun Zhu, Jianfei Chen Paper: https://arxiv.org/abs/2410.02367 **SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization** Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, Jianfei Chen Paper: https://arxiv.org/abs/2411.10958 **SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training** Jintao Zhang, Jia Wei, Haoxu Wang, Pengle Zhang, Xiaoming Xu, Haofeng Huang, Kai Jiang, Jianfei Chen, Jun Zhu Paper: https://arxiv.org/abs/2505.11594  *Note: [SageAttention2++](https://arxiv.org/pdf/2505.21136) achieves higher speed while maintaining the same accuracy performance.* ## Current Features <!-- This is a beta release of SageAttention2. We welcome any feedback on accuracy, performance issues, bugs, feature requests, or suggestions. Please feel free to open an issue or launch a pull request! --> + Optmized kernels for **Ampere, Ada and Hopper GPUs.** + INT8 quantization and smoothing for $QK^\top$ with support for varying granularities. + FP8 quantization for $PV$, and FP16 accumulator for FP8/FP16 $PV$. + Two-level accumulation strategy for $PV$ to improve accuracy in FP8 MMA and WGMMA. + Support `torch.compile` with non-cudagraphs mode and distributed inference. ## Project Updates - [2025-09-27]: 🎉 [SageAttention3](https://arxiv.org/abs/2505.11594) is accepted by NeurIPS 2025 as a **Spotlight** paper! - [2025-09-27]: The code of [SageAttention3](https://arxiv.org/abs/2505.11594) is released in this repository at [sageattention3_blackwell](./sageattention3_blackwell/). We would still greatly appreciate it if you could take a moment to fill out the Form in [Huggingface](https://huggingface.co/jt-zhang/SageAttention3). Please note that since SageAttention2 is more accurate, we still recommend using SageAttention2 for precision-sensitive applications. - [2025-07-01]: The code of [SageAttention2++](https://arxiv.org/pdf/2505.21136) is released in this repository. We would still greatly appreciate it if you could take a moment to fill out the Form in [Huggingface](https://huggingface.co/jt-zhang/SageAttention2_plus). Thank you very much!   - [2025-06-19]: [Sparse SageAttention1 API](https://github.com/jt-zhang/Sparse_SageAttention_API) and [Sparse SageAttention2 API](https://github.com/thu-ml/SpargeAttn) can compute attention with any block sparse pattern very fast. - [2025-05-02]: 🎉SageAttention2 and [SpargeAttn](https://github.com/thu-ml/SpargeAttn) are accepted by ICML 2025! - [2025-02-25]: 🔥 We release [SpargeAttn](https://github.com/thu-ml/SpargeAttn), a sparse attention based on SageAttention2, which could acclerate any model without training. - [2025-02-15]: 🔥 The compilation code is updated to support RTX5090! On RTX5090, SageAttention reaches 560T, 2.7x faster than FlashAttention2! - [2025-01-28]: 🔥⚡SageAttention is now available on Hopper GPUs (H100, H800, H20)! It matches the speed of FlashAttention3-FP8 but offers **much better accuracy!** | **FlashAttention2** | **FlashAttention3** | **FlashAttention3-FP8** | **SageAttention** | |----------------------|----------------------|----------------------|----------------------| |  |  |  | !
Excerpt of 11,315 characters
Read on GitHub
Jintao Zhang · @thu-ml, Tsinghua University · China
102
Haofeng Huang · Tsinghua University · China
34
Haoxu Wang · Tsinghua University · China
15
9
Guilherme Leobas
6
5
Nicholas Kwan
4
3
Tianqi Zhang (张天启) · bytedance · China
2
1
Qing Long · Freelance · China
1
1
1
1
Yichen Yan · Alibaba · China
1
1
DefTruth · @xlite-dev, @vipshop · China
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:12a90f794a87f395, topic:quantization, readme:quantization, desc:quantized
matched fp:12a90f794a87f395, topic:cuda
matched fp:12a90f794a87f395, topic:llm
matched fp:12a90f794a87f395, topic:video-generation