Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Runtimes and servers that execute model inference at speed and scale.
Signals: llm-inference, inference, inference-engine, model-serving, llm-serving, llama-cpp, vllm, triton-inference-server
411 results
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
Self-hosted huggingface mirror service. 自建huggingface镜像服务。
本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference
sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD
🔥🔥🔥TensorRT for YOLO26 (det/pose/seg/depth/obb/cls), YOLOv11,YOLOv10 YOLOv9,YOLOv8, ... 🚀🚀🚀 CUDA IS ALL YOU NEED. 🍎🍎🍎
LLM Client, Server API and UI
ONNX-TensorRT: TensorRT backend for ONNX
Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
A modern web interface for managing and interacting with vLLM servers (www.github.com/vllm-project/vllm). Supports both GPU and CPU modes, with special optimizations for macOS Apple Silicon and enterprise deployment on OpenShift/Kubernetes.
The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
GUI for upscaling videos with ONNX models, using NVIDIA TensorRT or DirectML
Access to Anthropic's safety-first language model APIs in C#
Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over OpenAI endpoints.
LLMRouter: An Open-Source Library for LLM Routing
AI-powered offensive security testing using autonomous agents, directly in your terminal.
LLMs as Copilots for Theorem Proving in Lean
Production-grade C++ edge AI engine for video analytics and on-device VLM across Sophon, Rockchip RKNN, and x86, with visual orchestration, real-time OSD, events, and reproducible benchmarks.
LLM plugin providing access to models running on an Ollama server
LLM plugin to access Google's Gemini family of models
WilmerAI is one of the oldest LLM semantic routers. It uses multi-layer prompt routing and complex workflows to allow you to not only create practical chatbots, but to extend any kind of application that connects to an LLM via REST API. Wilmer sits between your app and your many LLM APIs, so that you can manipulate prompts as needed.
High-performance MLX-based LLM inference engine for macOS with native Swift implementation
PyTorch Neural Network eXchange
24,545 repositories in the index in total.
