Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure
| Date | Stars |
|---|---|
| 2026-07-31 | 349 |
| 2026-08-01 | 349 |
| 2026-08-06 | 349 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
5.0
growth rate 0.00%/day
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/static/img/llmservingsim_full_primary_dark_transparent.png">
<img alt="LLMServingSim" src="docs/static/img/llmservingsim_full_primary_transparent.png" width="70%">
</picture>
</p>
<h3 align="center">
A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure
</h3>
<p align="center">
| <a href="https://llmservingsim.ai"><b>Website</b></a> | <a href="https://llmservingsim.ai/docs/getting-started/overview"><b>Documentation</b></a> | <a href="https://llmservingsim.ai/docs/contributor/welcome"><b>Contribute</b></a> | <a href="https://llmservingsim.ai/contact"><b>Contact</b></a> | <a href="https://llmservingsim.ai/changelog"><b>Changelog</b></a> |
</p>
We have built an LLMServingSim website to help you get started with the simulator. Please visit [llmservingsim.ai](https://llmservingsim.ai) for documentation, contribution guides, and team contact info.
## About
LLMServingSim is a cycle-level simulator for LLM serving infrastructure. It pairs a Python frontend that mirrors vLLM's continuous-batching scheduler with the ASTRA-Sim C++ analytical network backend, and drives both from per-hardware latency data captured by a vLLM-based layerwise profiler. The result is a unified environment for studying heterogeneous accelerators, disaggregated memory tiers (CPU / CXL / PIM), MoE routing, and multi-instance parallelism (TP / PP / EP / DP) end-to-end.
## Getting Started
```bash
git clone --recurse-submodules https://github.com/casys-kaist/LLMServingSim.git
cd LLMServingSim
./scripts/docker-sim.sh # launch the simulator container
./scripts/compile.sh # build ASTRA-Sim + Chakra
./serving/run.sh # run the example simulations
```
For installation details, container choices, configuration layout, CLI
flags, and the full set of example workloads, see the
[documentation](https://llmservingsim.ai/docs/getting-started/overview).
## Publications
**ISPASS 2026**
*LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure*
Jaehong Cho<sup>\*</sup>, Hyunmin Choi<sup>\*</sup>, Guseul Heo, Jongse Park (KAIST) [[Paper]](https://doi.org/10.1109/ISPASS69572.2026.00012)
<sup>\*</sup>Equal contribution
[](https://doi.org/10.5281/zenodo.18879965)
**CAL 2025**
*LLMServingSim2.0: A Unified Simulator for Heterogeneous Hardware and Serving Techniques in LLM Infrastructure*
Jaehong Cho, Hyunmin Choi, Jongse Park (KAIST) [[Paper]](https://doi.org/10.1109/LCA.2025.3628325)
**IISWC 2024**
*LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale*
Jaehong Cho, Minsu Kim, Hyunmin Choi, Guseul Heo, Jongse Park (KAIST) [[Paper]](https://doi.org/10.1109/IISWC63097.2024.00012)
[](https://doi.org/10.5281/zenodo.12803583)
## Citation
If you use LLMServingSim in your research, please cite:
```bibtex
@INPROCEEDINGS{11527300,
author={Cho, Jaehong and Choi, Hyunmin and Heo, Guseul and Park, Jongse},
booktitle={2026 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)},
title={{LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure}},
year={2026},
pages={1-14},
doi={10.1109/ISPASS69572.2026.00012}
}
@ARTICLE{11224567,
author={Cho, Jaehong and Choi, Hyunmin and Park, Jongse},
journal={IEEE Computer Architecture Letters},
title={{LLMServingSim2.0: A Unified Simulator for Heterogeneous Hardware and Serving
Techniques in LLM Infrastructure}},
year={2025},
volume={24},
number={02},
pages={361-364},
doi={10.1109/LCA.2025.3628325},
ISSN={1556-6064},
publisher={IEEE Computer Society},
address={Los Alamitos, CA, USA},
month=jul
}
@INPExcerpt of 4,398 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:914046d76f01c166, llm:Repository description: 'LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure' (Python project).
matched fp:914046d76f01c166, llm:Repository description: 'LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure' (Python project).
matched fp:914046d76f01c166, llm:Repository description: 'LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure' (Python project).