Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.
| Date | Stars |
|---|---|
| 2026-07-24 | 1133 |
| 2026-07-25 | 1133 |
| 2026-07-28 | 1133 |
| 2026-07-30 | 1133 |
| 2026-07-31 | 1132 |
| 2026-08-06 | 1132 |
Today
— stars today
This week
-1 stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<p align="center">
<img src="figures/logo.png" width="20%"> <br>
</p>
<div align="center">
<h1>LLM-Pruner</h1>
<div align="center">
<a href="https://opensource.org/licenses/Apache-2.0">
<img alt="License: Apache 2.0" src="https://img.shields.io/badge/License-Apache%202.0-4E94CE.svg">
</a>
<a href="https://pytorch.org/">
<img src="https://img.shields.io/badge/PyTorch-%3E=v1.7.1-EE4C2C.svg?style=flat-square" alt="PyTorch>=v1.7.1">
</a>
<a href="https://github.com/facebookresearch/llama">
<img src="https://img.shields.io/badge/LLMs-LLaMA-FFB000.svg?style=flat-square" alt="LLaMA">
</a>
<a href="https://github.com/facebookresearch/llama">
<img src="https://img.shields.io/badge/LLMs-Llama2-FAB093.svg?style=flat-square" alt="Llama-2">
</a>
<a href="https://github.com/facebookresearch/llama">
<img src="https://img.shields.io/badge/LLMs-Llama3&3.1-7CC217.svg?style=flat-square" alt="Llama-3">
</a>
<a href="https://github.com/lm-sys/FastChat">
<img src="https://img.shields.io/badge/LLMs-Vicuna-924E7D.svg?style=flat-square" alt="Vicuna">
</a>
<a href="https://huggingface.co/docs/transformers/model_doc/bloom">
<img src="https://img.shields.io/badge/LLMs-BLOOM-1A63BD.svg?style=flat-square" alt="BLOOM">
</a>
<a href="https://github.com/THUDM/ChatGLM-6B">
<img src="https://img.shields.io/badge/LLMs-chatGLM-6082B6.svg?style=flat-square" alt="chatGLM">
</a>
<a href="https://github.com/baichuan-inc/Baichuan-7B">
<img src="https://img.shields.io/badge/LLMs-Baichuan-18ac62.svg?style=flat-square" alt="Baichuan">
</a>
</div>
<h3>On the Structural Pruning of Large Language Models<h3>
:llama: :llama: :llama: :llama: :llama: Compress your LLMs to any size! :llama: :llama: :llama: :llama: :llama:
</div>
<p align="center">
<img width="100%" alt="image" src="figures/intro.png">
<img src="figures/LLaMA_example.png" width="100%"> <br>
</p>
## Introduction
> **[LLM-Pruner: On the Structural Pruning of Large Language Models](https://arxiv.org/abs/2305.11627)** [[arXiv]](https://arxiv.org/abs/2305.11627)
> *Xinyin Ma, Gongfan Fang, Xinchao Wang*
> *National University of Singapore*
#### Why LLM-Pruner
- [x] **Task-agnostic compression**: The compressed LLM should retain its original ability as a multi-task solver.
- [x] **Less training corpus**: In this work, we use only 50k publicly available samples (alpaca) to post-train the LLM.
- [x] **Efficient compression**: 3 minutes for pruning and 3 hours for post-training. (You can make it longer)
- [x] **Automatic structural pruning**: Pruning new LLMs with minimal human effort (In progress).
#### Supported LLMs:
- [x] [Llama-3.1](https://huggingface.co/collections/meta-llama/llama-31-669fc079a0c406a149a5738f)
- [x] [Llama-3](https://huggingface.co/collections/meta-llama/meta-llama-3-66214712577ca38149ebb2b6)
- [x] [Llama-2](https://github.com/horseee/LLM-Pruner#1-pruning-discovery-stage--estimation-stage)
- [x] [LLaMA](https://github.com/horseee/LLM-Pruner#1-pruning-discovery-stage--estimation-stage)
- [x] [BLOOM](https://github.com/horseee/LLM-Pruner/tree/main/examples#cherry_blossom-bloom)
- [x] [Vicuna](https://github.com/horseee/LLM-Pruner#llama-vicuna-pruning)
- [x] [Baichuan](https://github.com/horseee/LLM-Pruner/tree/main/examples#llama-baichuan-pruning)
- [x] [TinyLlama](https://github.com/jzhang38/TinyLlama)
#### Updates:
* July 27, 2024: :rocket: Support GQA! Now LLM-Pruner can work on Llama3 and Llama 3.1. We are still testing the pruning results of new LLMs (Llama3, Llama3.1, Gemma) and you can find the pruning results [here](https://github.com/horseee/LLM-Pruner/tree/main/more_results#more-results).
* August 30, 2023: LLM-Pruner now supports [BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom) :cherry_blossom:
* August 14, 2023: [Code](https://github.com/horseee/LLM-Pruner#2-post-training-recover-stage) and [results](https://github.com/horseee/LLM-Pruner#2-post-training-recover-stage) fExcerpt of 21,651 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:122a1a3065fc2d07, topic:llm, topic:llama, topic:language-model
matched fp:122a1a3065fc2d07, topic:pruning, desc:pruning, readme:pruning
