Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
| Date | Stars |
|---|---|
| 2026-07-31 | 1472 |
| 2026-08-06 | 1472 |
| 2026-08-10 | 1499 |
| 2026-08-18 | 1516 |
| 2026-08-19 | 1525 |
| 2026-08-20 | 1528 |
| 2026-08-21 | 1533 |
| 2026-08-22 | 1533 |
| 2026-08-23 | 1534 |
| 2026-08-24 | 1536 |
| 2026-08-25 | 1540 |
| 2026-08-26 | 1546 |
| 2026-08-27 | 1548 |
| 2026-08-28 | 1550 |
| 2026-08-29 | 1551 |
| 2026-08-30 | 1552 |
| 2026-08-31 | 1555 |
| 2026-09-01 | 1556 |
| 2026-09-02 | 1557 |
| 2026-09-03 | 1556 |
| 2026-09-04 | 1556 |
| 2026-09-05 | 1557 |
| 2026-09-06 | 1560 |
| 2026-09-08 | 1563 |
| 2026-09-09 | 1562 |
| 2026-09-10 | 1567 |
| 2026-09-11 | 1567 |
| 2026-09-12 | 1570 |
| 2026-09-13 | 1570 |
| 2026-09-15 | 1571 |
| 2026-09-16 | 1575 |
| 2026-09-17 | 1576 |
| 2026-09-18 | 1580 |
| 2026-09-19 | 1587 |
| 2026-09-20 | 1588 |
| 2026-09-21 | 1590 |
| 2026-09-22 | 1589 |
| 2026-09-23 | 1591 |
| 2026-09-24 | 1592 |
| 2026-09-25 | 1593 |
| 2026-09-26 | 1595 |
| 2026-09-27 | 1596 |
| 2026-09-28 | 1599 |
| 2026-09-29 | 1601 |
| 2026-09-30 | 1604 |
| 2026-10-01 | 1607 |
| 2026-10-02 | 1607 |
| 2026-10-03 | 1608 |
| 2026-10-04 | 1608 |
| 2026-10-05 | 1610 |
| 2026-10-06 | 1614 |
| 2026-10-07 | 1612 |
| 2026-10-08 | 1614 |
| 2026-10-09 | 1617 |
| 2026-10-10 | 1618 |
| 2026-10-12 | 1618 |
| 2026-10-13 | 1619 |
| 2026-10-14 | 1622 |
| 2026-10-15 | 1624 |
| 2026-10-16 | 1624 |
| 2026-10-17 | 1625 |
| 2026-10-18 | 1626 |
| 2026-10-19 | 1626 |
| 2026-10-20 | 1628 |
| 2026-10-21 | 1630 |
| 2026-10-22 | 1632 |
| 2026-10-23 | 1635 |
| 2026-10-24 | 1637 |
| 2026-10-25 | 1638 |
| 2026-10-26 | 1638 |
| 2026-10-28 | 1639 |
| 2026-10-29 | 1641 |
| 2026-10-30 | 1642 |
| 2026-10-31 | 1644 |
| 2026-11-01 | 1644 |
| 2026-11-02 | 1644 |
| 2026-11-03 | 1646 |
| 2026-11-04 | 1646 |
Today
— stars today
This week
+7 stars this week
This month
+36 stars this month
Momentum
92.0
growth rate 0.43%/day
# vllm-mlx
**Continuous batching + OpenAI + Anthropic APIs in one server. Native Apple Silicon inference.**
**Read this in other languages:** [English](README.md) · [Español](README.es.md) · [Français](README.fr.md) · [中文](README.zh.md)
[](https://pypi.org/project/vllm-mlx/)
[](https://pepy.tech/projects/vllm-mlx)
[](https://www.python.org/downloads/)
[](LICENSE)
[](https://support.apple.com/en-us/HT211814)
[](https://github.com/waybarrios/vllm-mlx)
---
## What is vllm-mlx?
A vLLM-style inference server for Apple Silicon Macs. Unlike `Ollama` or `mlx-lm` used directly, it ships **continuous batching, paged KV cache, prefix caching, and SSD-tiered cache**, and exposes **both OpenAI `/v1/*` and Anthropic `/v1/messages`** from a single process. Run LLMs, vision models, audio, and embeddings on Metal with unified memory, no conversion step.
## Quick start
New users: follow the [isolated release install](docs/getting-started/installation.md#install-a-release)
and [first-response check](docs/getting-started/quickstart.md#first-response).
The example below assumes that environment is activated. Initial model
download/loading time depends on the machine and connection.
The version below is this walkthrough's pinned reference release, not an
automatically updated latest version.
```bash
python -m pip install 'vllm-mlx==0.4.1'
vllm-mlx serve mlx-community/Llama-3.2-3B-Instruct-4bit --host 127.0.0.1 --port 8000
```
After the first response, restart with `--continuous-batching` to try
[continuous batching and its cache options](docs/guides/continuous-batching.md).
The minimal command above uses the default engine without that flag.
**OpenAI SDK:**
Install `openai` in your Python client environment first; it is not installed
by the server package. See the [client setup](docs/getting-started/quickstart.md#option-1-openai-compatible-server).
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
r = client.chat.completions.create(model="mlx-community/Llama-3.2-3B-Instruct-4bit", messages=[{"role": "user", "content": "Hi!"}])
print(r.choices[0].message.content)
```
**Anthropic SDK / Claude Code:**
```bash
export ANTHROPIC_BASE_URL=http://localhost:8000
export ANTHROPIC_API_KEY=not-needed
claude
```
## Coding CLI compatibility
**Validated with OpenCode, pi, Codex, Claude Code, GitHub Copilot CLI, Cline CLI,
and OpenClaw's embedded agent.** All seven completed a streamed tool interaction
and an exact file edit in one local run with Qwen3.8-27B-4bit on September 19,
2026. Results apply to the tested client versions and settings.
See the [validated CLI matrix and setup guide](docs/guides/client-acceptance.md#validated-cli-versions)
for versions, API transports, reproduction commands, and coverage limits.
## Features
### APIs
- **OpenAI-compatible**: `/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`, `/v1/rerank`, `/v1/responses`
- **Anthropic-compatible**: `/v1/messages` (streaming, tool use, system prompts)
- **MCP Tool Calling**: 19 parsers (OpenAI, Anthropic, Gemini, Qwen, DeepSeek, Gemma, and more)
- **Structured output**: JSON Schema via `response_format` (lm-format-enforcer)
### Throughput & memory
- **Continuous batching**: high throughput for concurrent requests
- **Paged KV cache**: memory-efficient with prefix sharing
- **SSD-tiered KV cache**: spill prefix cache to disk for long-context agents (`--ssd-cache-dir`)
- **Warm prompts**: preload popular prefixes at startup (`--warm-prompts`) for 1.3-2.25x TTFT
- **Prefix cache**: trie-based, shared acExcerpt of 14,119 characters
Read on GitHub
204
Wayner Barrios · Dartmouth · United States
157
Jan Hilgard · Czech Republic
48
25
14
12
11
6
5
4
4
3
3
2
2
2
2
2
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:2918202d39e325df, topic:text-to-speech, topic:tts, topic:stt
matched fp:2918202d39e325df, topic:inference, topic:vllm, readme:inference server