Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ACL 2026] Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
| Date | Stars |
|---|---|
| 2026-07-31 | 365 |
| 2026-08-01 | 366 |
| 2026-08-06 | 366 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<h1 align="center">Awesome KV Cache Optimization</h1>
<div align="center">
</div>
<div align="center">
**[<a href="https://aclanthology.org/2026.findings-acl.1916/">ACL Anthology</a>]**
**[<a href="https://www.linkedin.com/feed/update/urn:li:activity:7384388868407529472/">LinkedIn</a>]**
**[<a href="https://mp.weixin.qq.com/s/466l3Dh4xD4LRLR8vSOc8Q">机器之心</a>]**
</div>
This repository is for our survey paper (**ACL 2026 Findings**):
> **[Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization](https://aclanthology.org/2026.findings-acl.1916/)**
> *[Jiantong Jiang](https://jjiantong.github.io/)<sup>1</sup>, [Peiyu Yang](https://ypeiyu.github.io/)<sup>1\*</sup>, [Rui Zhang](https://www.ruizhang.info/)<sup>2</sup>, [Feng Liu](https://fengliu90.github.io/)<sup>1</sup>
> <sup>1</sup>The University of Melbourne, <sup>2</sup>Huazhong University of Science and Technology
---
This repository aims to record papers of system-aware, serving-time, KV-centric optimization methods that improve system metrics without retraining or architecture modification (which we call this scope ***sKis***). We systematize recent advances via a **system behavior-oriented taxonomy**, organizing existing efforts into three behavioral dimensions:\
🔷 **Temporal** — when is KV cache accessed or computed?\
🔷 **Spatial** — where is KV cache placed and migrated?\
🔷 **Structural** — how is KV cache represented and managed?
🧠 Grounded in this taxonomy, we analyze **cross-behavior co-design affinity** and **behavior–objective effects**, revealing overlooked regions and concrete open challenges.
<p align="center">
<img src="assets/awesome-cover.png" width="90%" style="align:center;"/>
</p>
### News
- 2026-06: 🔥 51 new papers were added.
- 2026-04: 🎉 Our survey was accepted to ACL 2026.
- 2026-01: 📰 Our survey was featured by 机器之心.
- 2025-11: ⭐ 100-star milestone! Thanks to the amazing support from the community!
- 2025-10: 📝 The survey was available on TechRxiv.
### Contributing
The repository is **still under active development** and will be updated regularly.
🙋 If you would like to include your paper in this survey and repository, please feel free to submit a pull request. You can generate the markdown row for each paper by filling in the first part of ```generate.py```
and running ```python generate.py```. Alternatively, you can open an issue with the paper's title and a brief summary highlighting its key techniques. You can also contact us via email.
🙋🏻♀️ Please let us know if you find out a mistake or have any suggestions! We greatly appreciate your feedback regarding this repository or survey!
🌟 If you find this resource helpful for your work, please consider giving us a star and citing our [research](#citation).
---
<a name="readme-index"></a>
## Quick Index
- [Temporal — Execution \& Scheduling](#temporal--execution--scheduling)
- [KV-Centric Scheduling (KVS)](#kv-centric-scheduling)
- [Pipelining \& Overlapping (OVLP)](#pipelining--overlapping)
- [Hardware-Aware Execution (HAE)](#hardware-aware-execution)
- [Spatial — Placement \& Migration](#spatial--placement--migration)
- [Memory Hierarchy KV Orchestration (MHO)](#memory-hierarchy-kv-orchestration)
- [Compute Device KV Orchestration (CDO)](#compute-device-kv-orchestration)
- [Structural — Representation \& Retention](#structural--representation--retention)
- [KV Cache Compression (KVCC)](#kv-cache-compression) (including quantization, low-rank approximation, and structural compression, and codec-based compression)
- [KV Cache Retention ManagemenExcerpt of 119,690 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:4704161b17c2865a, topic:llm-inference, topic:llm-serving, desc:model serving