Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Runtimes and servers that execute model inference at speed and scale.
Signals: llm-inference, inference, inference-engine, model-serving, llm-serving, llama-cpp, vllm, triton-inference-server
411 results
Deep learning gateway on Raspberry Pi and other edge devices
模型部署白皮书(CUDA|ONNX|TensorRT|C++)🚀🚀🚀
A very fast neural network computing framework optimized for mobile platforms.QQ group: 676883532 【验证信息输：绝影】
use ncnn in Android and iOS, ncnn is a high-performance neural network inference framework optimized for the mobile platform
Apply a Style Transfer Neural Network in real time with Unreal Engine 5 leveraging ONNX Runtime.
FPGA-based neural network inference project with an end-to-end approach (from training to implementation to deployment)
Zant simplifies the deployment and optimization of neural networks on microprocessors
HLS based Deep Neural Network Accelerator Library for Xilinx Ultrascale+ MPSoCs
MCP Server to Use HuggingFace spaces, easy configuration and Claude Desktop mode.
Gemma 2 optimized for your local machine.
No description
Turn Antigravity / codex / github copilot into Anthropic & Openai API compatible server. Usable with Claude Code / Xcode etc.
Access models from OpenAI, Groq, local Ollama, and others by setting llm-router as Cursor's Base URL
The platform for AI computers
Vercel AI Provider for running LLMs locally using Ollama
Run full Kimi K3 on a single device. And an OpenAI-compatible API server for local chat and coding agents.
Curated collection of AI inference engineering resources — LLM serving, GPU kernels, quantization, distributed inference, and production deployment. Compiled from the AER Labs community.
[ICLR2025 Spotlight] MagicPIG: LSH Sampling for Efficient LLM Generation
Work with LLMs on a local environment using containers
Plugin for LLM adding support for the GPT4All collection of models
Demonstration of running a native LLM on Android device.
Minimal yet performant LLM examples in pure JAX
No description
24,545 repositories in the index in total.
