Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
| Date | Stars |
|---|---|
| 2026-07-31 | 18326 |
| 2026-08-01 | 18369 |
| 2026-08-06 | 18369 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
60.0
growth rate 0.00%/day
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/images/icon-rounded-dark.svg" width="140">
<source media="(prefers-color-scheme: light)" srcset="docs/images/icon-rounded-light.svg" width="140">
<img alt="oMLX" src="docs/images/icon-rounded-light.svg" width="140">
</picture>
</p>
<h1 align="center">oMLX</h1>
<p align="center"><b>LLM inference, optimized for your Mac</b><br>Continuous batching and tiered KV caching, managed directly from your menu bar.</p>
<p align="center">
<a href="https://www.buymeacoffee.com/jundot"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" height="40"></a>
</p>
<p align="center">
<img src="https://img.shields.io/badge/license-Apache%202.0-blue" alt="License">
<img src="https://img.shields.io/badge/python-3.11--3.13-green" alt="Python 3.11-3.13">
<img src="https://img.shields.io/badge/platform-Apple%20Silicon-black?logo=apple" alt="Apple Silicon">
</p>
<p align="center">
<a href="mailto:[email protected]">[email protected]</a> · <a href="https://omlx.ai/me">https://omlx.ai/me</a>
</p>
<p align="center">
<a href="#install">Install</a> ·
<a href="#quickstart">Quickstart</a> ·
<a href="#features">Features</a> ·
<a href="#models">Models</a> ·
<a href="#cli-configuration">CLI Configuration</a> ·
<a href="https://omlx.ai/benchmarks">Benchmarks</a> ·
<a href="https://omlx.ai">oMLX.ai</a>
</p>
<p align="center">
<b>English</b> ·
<a href="README.zh.md">中文</a> ·
<a href="README.ko.md">한국어</a> ·
<a href="README.ja.md">日本語</a>
</p>
---
<p align="center">
<img src="docs/images/omlx_dashboard.png" alt="oMLX Admin Dashboard" width="800">
</p>
> *Every LLM server I tried made me choose between convenience and control. I wanted to pin everyday models in memory, auto-swap heavier ones on demand, set context limits - and manage it all from a menu bar.*
>
> *oMLX persists KV cache across a hot in-memory tier and cold SSD tier - even when context changes mid-conversation, all past context stays cached and reusable across requests, making local LLMs practical for real coding work with tools like Claude Code. That's why I built it.*
## Install
### macOS App
Download the `.dmg` from [Releases](https://github.com/jundot/omlx/releases), drag to Applications, done. The app includes in-app auto-update, so future upgrades are just one click. The macOS app also installs a lightweight `~/.omlx/bin/omlx` CLI shim so terminal commands and Apple Shortcuts can control the app-managed server.
### Homebrew
```bash
brew tap jundot/omlx https://github.com/jundot/omlx
brew install omlx
# Upgrade to the latest version
brew update && brew upgrade omlx
# Run as a background service (auto-restarts on crash)
omlx start
# Optional: MCP (Model Context Protocol) support
/opt/homebrew/opt/omlx/libexec/bin/pip install mcp
```
Optional GLM-5.2 / MiniMax M3 native custom kernels currently require a HEAD build:
```bash
brew install omlx --HEAD --with-custom-kernel
```
### From Source
```bash
git clone https://github.com/jundot/omlx.git
cd omlx
pip install -e . # Core only
pip install -e ".[mcp]" # With MCP (Model Context Protocol) support
# GLM-5.2 / MiniMax M3 / Qwen3.5 native custom kernels (strongly recommended
# if you serve those families -- see note below)
OMLX_WITH_CUSTOM_KERNEL=1 pip install -e .
```
Requires macOS 15.0+ (Sequoia), Python 3.11–3.13, and Apple Silicon (M1/M2/M3/M4).
> **Note on native custom kernels:** a plain `pip install -e .` does NOT build
> them, and the affected model families then silently fall back to much slower
> generic paths -- for GLM-5.2 the fused DSA prefill is roughly 30x faster with
> the kernels (measured 845 vs ~29 tok/s on an M3 Ultra), and the fallback also
> uses more memory (#2137). Building them requires the Metal toolchain, which
> Command Line Tools alone do not provide (`xcrun: error: unable to find utility
> "metal"`): install full XcExcerpt of 18,428 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:bdb813b0fd5a2550, topic:llm
matched fp:bdb813b0fd5a2550, desc:inference server, desc:continuous batching
