Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Big & Small LLMs working together
| Date | Stars |
|---|---|
| 2026-07-31 | 1342 |
| 2026-08-06 | 1345 |
Today
+3 stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Where On-Device and Cloud LLMs Meet
[](https://discord.gg/jfJyxXwFVa)
_What is this?_ Minions is a communication protocol that enables small on-device models to collaborate with frontier models in the cloud. By only reading long contexts locally, we can reduce cloud costs with minimal or no quality degradation. This repository provides a demonstration of the protocol. Get started below or see our paper and blogpost below for more information.
Paper: [Minions: Cost-efficient Collaboration Between On-device and Cloud
Language Models](https://arxiv.org/pdf/2502.15964)
Minions Blogpost: https://hazyresearch.stanford.edu/blog/2025-02-24-minions
Secure Minions Chat Blogpost: https://hazyresearch.stanford.edu/blog/2025-05-12-security
## Table of Contents
> **Looking for Secure Minions Chat?** If you're interested in our end-to-end encrypted and chat system, please see the [Secure Minions Chat README](secure/README.md) for detailed setup and usage instructions.
- [Setup](#setup)
- [Step 1: Clone and Install](#step-1-clone-the-repository-and-install-the-python-package)
- [Step 2: Install a Local Model Server](#step-2-install-a-server-for-running-the-local-model)
- [Step 3: Set Cloud LLM API Keys](#step-3-set-your-api-key-for-at-least-one-of-the-following-cloud-llm-providers)
- [Minions Demo Application](#minions-demo-application)
- [Minions WebGPU App](#minions-webgpu-app)
- [Example Code](#example-code-minion-singular)
- [Minion (Singular)](#example-code-minion-singular)
- [Minions (Plural)](#example-code-minions-plural)
- [Python Notebook](#python-notebook)
- [Docker Support](#docker-support)
- [Command Line Interface](#cli)
- [Secure Minions Local-Remote Protocol](#secure-minions-local-remote-protocol)
- [Secure Minions Chat](#secure-minions-chat)
- [Apps](#apps)
- [Inference Estimator](#inference-estimator)
- [Command Line Usage](#command-line-usage)
- [Python API Usage](#python-api-usage)
- [Miscellaneous Setup](#miscellaneous-setup)
- [Using Azure OpenAI](#using-azure-openai-with-minions)
- [Maintainers](#maintainers)
## Setup
_We have tested the following setup on Mac and Ubuntu with Python 3.10-3.11_ (Note: Python 3.13 is not supported)
<details>
<summary>Optional: Create a virtual environment with your favorite package manager (e.g. conda, venv, uv)</summary>
```python
conda create -n minions python=3.11
```
</details><br>
**Step 1:** Clone the repository and install the Python package.
```bash
git clone https://github.com/HazyResearch/minions.git
cd minions
pip install -e . # installs the minions package in editable mode
```
_note_: for optional MLX-LM install the package with the following command:
```bash
pip install -e ".[mlx]"
```
_note_: for secure minions chat, install the package with the following command:
```bash
pip install -e ".[secure]"
```
_note_: for optional Cartesia-MLX install, pip install the basic package and then follow the instructions below.
**Step 2:** Install a server for running the local model.
We support three servers for running local models: `lemonade`, `ollama`, and `tokasaurus`. You need to install at least one of these.
- You should use `ollama` if you do not have access to NVIDIA/AMD GPUs. Install `ollama` following the instructions [here](https://ollama.com/download). To enable Flash Attention, run
`launchctl setenv OLLAMA_FLASH_ATTENTION 1` and, if on a mac, restart the ollama app.
- You should use `lemonade` if you have access to local AMD CPUs/GPUs/NPUs. Install `lemonade` following the instructions [here](https://lemonade-server.ai/).
- See the following for supported APU configurations: https://ryzenai.docs.amd.com/en/latest/llm/overview.html#supported-configurations
- After installing `lemonade` make sure to launch the lemonade server. This can be done via the [one-click Windows GUI](https://lemoExcerpt of 24,940 characters
Read on GitHub
Avanika Narayan
530
45
44
26
Cursor Agent
24
Ignasi
19
15
11
7
6
5
5
4
3
3
3
2
Jacob Howard · Colombia
2
Doiiars · China
2
2
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:7bbf767dc71d39ec, llm:Repository description: "Big & Small LLMs working together" (HazyResearch/minions). Language: Python.
matched fp:7bbf767dc71d39ec, llm:Repository description: "Big & Small LLMs working together" (HazyResearch/minions). Language: Python.
matched fp:7bbf767dc71d39ec, llm:Repository description: "Big & Small LLMs working together" (HazyResearch/minions). Language: Python.