Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Unified LLM gateway with weighted load balancing, observability & cost tracking. 统一的 LLM 网关，提供权重负载均衡、可观测性与费用追踪。
| Date | Stars |
|---|---|
| 2026-07-24 | 310 |
| 2026-07-25 | 310 |
| 2026-07-28 | 310 |
| 2026-07-30 | 310 |
| 2026-07-31 | 314 |
| 2026-08-06 | 314 |
Today
— stars today
This week
+4 stars this week
This month
— stars this month
Momentum
4.0
growth rate 1.29%/day
# LLMIO
English | [中文](README_cn.md)
LLMIO is a Go-based LLM load‑balancing gateway that provides a unified REST API, weighted scheduling, observability, and a modern admin UI for LLM clients (openclaw / claude code / codex / gemini cli / cherry studio / open webui). It helps you integrate OpenAI, Anthropic, Gemini, and other model capabilities in a single service.
**QQ group: 1083599685**
## Architecture
## Features
- **Unified API**: Compatible with OpenAI Chat Completions, OpenAI Responses, Gemini Native, and Anthropic Messages. Supports both streaming and non‑streaming passthrough.
- **Weighted scheduling**: `balancers/` provides two strategies (random by weight / priority by weight). You can route based on tool calling, structured output, and multimodal capability.
- **Admin Web UI**: React + TypeScript + Tailwind + Vite console for providers, models, associations, logs, and metrics.
- **Rate limiting & failure handling**: Built‑in rate‑limit fallback and provider connectivity checks for fault isolation.
- **Local persistence**: Pure Go SQLite (`db/llmio.db`) for config and request logs, ready to use out of the box.
- **Session tracking**: Pass `session_id` in any request body (works with `extra_body` in OpenAI SDK) to tag logs with a session identifier. Filter and search by `session_id` in the admin UI or via `GET /api/logs?session_id=`.
- **Observability**: Every request is recorded with TraceID, latency breakdown (proxy / first-chunk / completion time), TPS, token usage (input / cached / output), and optional full IO logging. Per-request cost is calculated from configurable per-million-token prices (CNY / USD) and shown in the log detail view alongside provider and model metadata.
## Deployment
### Docker Compose (Recommended)
```yaml
services:
llmio:
image: atopos31/llmio:latest
ports:
- 7070:7070
volumes:
- ./db:/app/db
environment:
- GIN_MODE=release
- TOKEN=<YOUR_TOKEN>
- TZ=Asia/Shanghai
```
```bash
docker compose up -d
```
### Docker
```bash
docker run -d \
--name llmio \
-p 7070:7070 \
-v $(pwd)/db:/app/db \
-e GIN_MODE=release \
-e TOKEN=<YOUR_TOKEN> \
-e TZ=Asia/Shanghai \
atopos31/llmio:latest
```
### Local Run
Download the release package for your OS/arch from [releases](https://github.com/atopos31/llmio/releases) (version > 0.5.13). Example for linux amd64:
```bash
wget https://github.com/atopos31/llmio/releases/download/v0.5.13/llmio_0.5.13_linux_amd64.tar.gz
```
Extract:
```bash
tar -xzf ./llmio_0.5.13_linux_amd64.tar.gz
```
Start:
```bash
GIN_MODE=release TOKEN=<YOUR_TOKEN> ./llmio
```
The service will create `./db/llmio.db` in the current directory as the SQLite persistence file.
## Environment Variables
| Variable | Description | Default | Notes |
|---|---|---|---|
| `TOKEN` | Console login and API auth for `/openai` `/anthropic` `/gemini` `/v1` | None | Required for public access |
| `GIN_MODE` | Gin runtime mode | `debug` | Use `release` in production |
| `LLMIO_SERVER_PORT` | Server listen port | `7070` | Service listen port |
| `TZ` | Timezone for logs and scheduling | Host default | Recommend explicit setting in containers (e.g. `Asia/Shanghai`) |
| `DB_VACUUM` | Run SQLite VACUUM on startup | Disabled | Set to `true` to reclaim space |
## Development
Clone:
```bash
git clone https://github.com/atopos31/llmio.git
cd llmio
```
Build frontend (pnpm required):
```bash
make webui
```
Run backend (Go >= 1.26.1):
```bash
TOKEN=<YOUR_TOKEN> make run
```
Web UI: `http://localhost:7070/`
## API Endpoints
LLMIO provides a multi‑provider REST API with the following endpoints:
| Provider | Path | Method | Description | Auth |
|---|---|---|---|---|
| OpenAI | `/openai/v1/models` | GET | List available models | Bearer Token |
| OpenAI | `/openai/v1/chat/completions` | POST | Create chat completion | Bearer Token |
| OpenAI | `/openai/v1/responses` | POST | Create response | Bearer Token |
| AntExcerpt of 7,495 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:83d788f4e390db19, topic:transformer, topic:qwen
matched fp:83d788f4e390db19, topic:ai-gateway, desc:llm gateway, readme:unified api