Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
[ICLR 2026] ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
| Date | Stars |
|---|---|
| 2026-07-31 | 327 |
| 2026-08-02 | 326 |
| 2026-08-06 | 326 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# ParoQuant **Pairwise Rotation Quantization for Efficient Reasoning LLM Inference** <p> <a href="https://arxiv.org/abs/2511.10645"><img src="https://img.shields.io/badge/arXiv-2511.10645-b31b1b.svg" alt="Paper"></a> <a href="https://paroquant.z-lab.ai"><img src="https://img.shields.io/badge/Blog-ParoQuant-blue" alt="Blog"></a> <a href="https://huggingface.co/collections/z-lab/paroquant"><img src="https://img.shields.io/badge/%F0%9F%A4%97-Models-yellow" alt="Models"></a> <a href="https://pypi.org/project/paroquant/"><img src="https://img.shields.io/pypi/v/paroquant" alt="PyPI"></a> </p> State-of-the-art INT4 quantization for LLMs. ParoQuant uses learned pairwise rotations to suppress weight outliers, closing the accuracy gap with FP16 while running at near-AWQ speed. Supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). ## Quick Start ### oMLX (Apple Silicon) On Apple Silicon, we recommend serving ParoQuant models using [oMLX](https://github.com/jundot/omlx). Refer to the [docs](https://omlx.ai) for more details. ### Installation ```bash # NVIDIA GPU (CUDA 12.9) pip install "paroquant[vllm]" # NVIDIA GPU (CUDA 13.0) pip install "paroquant[vllm]" "vllm==0.19.1" \ --extra-index-url https://wheels.vllm.ai/0.19.1/cu130 \ --extra-index-url https://download.pytorch.org/whl/cu130 # Apple Silicon pip install "paroquant[mlx]" ``` Pick a model from our [Hugging Face collection](https://huggingface.co/collections/z-lab/paroquant): ```bash export MODEL=z-lab/Qwen3.5-4B-PARO ``` ### Interactive Chat ```bash python -m paroquant.cli.chat --model $MODEL ``` ### OpenAI-Compatible API Server For vLLM, you can directly use `vllm serve` to serve ParoQuant models: ```bash vllm serve $MODEL --port 8000 ``` For other frameworks: ```bash python -m paroquant.cli.serve --model $MODEL --port 8000 ``` For MLX, add `--vlm` if you wish to load the VLM components and use the model's multimodal features. For vLLM, VLM components are loaded by default and can be skipped with the server argument `--language-model-only`. ### Docker (NVIDIA GPU) > [!NOTE] > The following commands map the local cache directory to the container in order to persist kernel cache across runs. Remove `-v ...` to disable this behaviour. ```bash # Interactive chat docker run --pull=always --rm -it --gpus all --ipc=host \ -v $HOME/.cache/paroquant:/root/.cache/paroquant \ ghcr.io/z-lab/paroquant:chat --model $MODEL # API server (port 8000) docker run --pull=always --rm -it --gpus all --ipc=host -p 8000:8000 \ -v $HOME/.cache/paroquant:/root/.cache/paroquant \ ghcr.io/z-lab/paroquant:serve --model $MODEL ``` ## Models All models are available on [Hugging Face](https://huggingface.co/collections/z-lab/paroquant). Swap the model name in the commands above to try any of them. **Gemma 4** | Model | Checkpoint | | ------------------ | --------------------------------------------------------------------------------------- | | gemma-4-31B-it | [`z-lab/gemma-4-31B-it-PARO`](https://huggingface.co/z-lab/gemma-4-31B-it-PARO) | | gemma-4-12B-it | [`z-lab/gemma-4-12B-it-PARO`](https://huggingface.co/z-lab/gemma-4-12B-it-PARO) | | gemma-4-26B-A4B-it | [`z-lab/gemma-4-26B-A4B-it-PARO`](https://huggingface.co/z-lab/gemma-4-26B-A4B-it-PARO) | | gemma-4-E4B-it | [`z-lab/gemma-4-E4B-it-PARO`](https://huggingface.co/z-lab/gemma-4-E4B-it-PARO) | | gemma-4-E2B-it | [`z-lab/gemma-4-E2B-it-PARO`](https://huggingface.co/z-lab/gemma-4-E2B-it-PARO) | **Qwen3.6** | Model | Checkpoint | | --------------- | --------------------------------------------------------------------------------- | | Qwen3.6-27B | [`z-lab/Qwen3.6-27B-PARO`](https://huggingface.co/z-lab/Qwen3.6-27B-PARO) | | Qwen3.6-35B-A3B | [`z-lab/Qwen3.6-35B-A3B-PARO
Excerpt of 7,827 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:31b9f7f25a616869, desc:quantization