Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A high-performance API server that provides OpenAI-compatible endpoints for MLX models. Developed using Python and powered by the FastAPI framework, it provides an efficient, scalable, and user-friendly solution for running MLX-based vision and language models locally with an OpenAI-compatible interface.
| Date | Stars |
|---|---|
| 2026-07-24 | 352 |
| 2026-07-25 | 353 |
| 2026-07-28 | 355 |
| 2026-07-30 | 356 |
| 2026-08-06 | 356 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
5.0
growth rate 0.00%/day
# mlx-openai-server
[](LICENSE)
[](https://www.python.org/downloads/release/python-3110/)
OpenAI-compatible API server for local MLX models on Apple Silicon. It serves text, multimodal, image, embedding, and Whisper models through familiar OpenAI SDK endpoints.
> Requires macOS on Apple Silicon and Python 3.11+.
## Contents
- [Feature Launch](#feature-launch)
- [Install](#install)
- [Start a Server](#start-a-server)
- [API Usage](#api-usage)
- [Server Options](#server-options)
- [Multi-Model Config](#multi-model-config)
- [Long Context and Metal OOM](#long-context-and-metal-oom)
- [Advanced LM Options](#advanced-lm-options)
- [Troubleshooting](#troubleshooting)
- [Examples and Demos](#examples-and-demos)
## Feature Launch
Darwin 36B Opus is now available in MLX format for local text inference:
- Original model: [FINAL-Bench/Darwin-36B-Opus](https://huggingface.co/FINAL-Bench/Darwin-36B-Opus)
- MLX text-only 8-bit conversion: [GiaHuy/Darwin-36B-Opus-mlx-text-only-8bit](https://huggingface.co/GiaHuy/Darwin-36B-Opus-mlx-text-only-8bit)
Launch it with the required reasoning and tool-call parsers:
```bash
mlx-openai-server launch --model-path Darwin-36B-Opus-mlx-text-only-8bit --reasoning-parser qwen3_moe --tool-call-parser qwen3_coder --debug --served-model-name Darwin-36B-Opus
```
Darwin 36B Opus should be served with both `--reasoning-parser qwen3_moe` and `--tool-call-parser qwen3_coder`; without them, reasoning and tool-call output will not be parsed correctly.
## Install
```bash
python3.11 -m venv .venv
source .venv/bin/activate
uv pip install mlx-openai-server
```
Install from GitHub instead:
```bash
uv pip install git+https://github.com/cubist38/mlx-openai-server.git
```
Whisper transcription also needs ffmpeg:
```bash
brew install ffmpeg
```
## Start a Server
Text model:
```bash
mlx-openai-server launch \
--model-type lm \
--model-path mlx-community/Qwen3-Coder-Next-4bit \
--reasoning-parser qwen3_moe \
--tool-call-parser qwen3_coder
```
Point OpenAI-compatible clients to:
```text
http://localhost:8000/v1
```
Use any non-empty API key, for example `not-needed`.
Common launch modes:
```bash
# Multimodal text/image/audio
mlx-openai-server launch \
--model-type multimodal \
--model-path <mlx-vlm-model>
# Image generation
mlx-openai-server launch \
--model-type image-generation \
--model-path <flux-or-qwen-image-model> \
--config-name flux-dev \
--quantize 8
# Image editing
mlx-openai-server launch \
--model-type image-edit \
--model-path <flux-or-qwen-image-edit-model> \
--config-name flux-kontext-dev \
--quantize 8
# Embeddings
mlx-openai-server launch \
--model-type embeddings \
--model-path <embedding-model>
# Whisper transcription
mlx-openai-server launch \
--model-type whisper \
--model-path mlx-community/whisper-large-v3-mlx
```
Supported model types:
| Type | Backend | Endpoint family |
|------|---------|-----------------|
| `lm` | `mlx-lm` | chat, responses |
| `multimodal` | `mlx-vlm` | chat, responses |
| `image-generation` | `mflux` | image generation |
| `image-edit` | `mflux` | image editing |
| `embeddings` | `mlx-embeddings` | embeddings |
| `whisper` | `mlx-whisper` | audio transcription |
Image `--config-name` values:
- Generation: `flux-schnell`, `flux-dev`, `flux-krea-dev`, `flux2-klein-4b`, `flux2-klein-9b`, `qwen-image`, `z-image-turbo`, `fibo`
- Editing: `flux-kontext-dev`, `flux2-klein-edit-4b`, `flux2-klein-edit-9b`, `qwen-image-edit`
## API Usage
### Chat
```python
import openai
client = openai.OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
response = client.chat.completions.create(
model="mlx-community/Qwen3-Coder-Next-4bit",
messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(response.choices[0].message.content)
```
### Vision
```python
importExcerpt of 16,131 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:3fb6776527b9e61e, topic:image-generation, topic:flux, readme:image generation
matched fp:3fb6776527b9e61e, topic:speech-recognition, topic:whisper, readme:transcription
matched fp:3fb6776527b9e61e, topic:tool-calling