Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Runtimes and servers that execute model inference at speed and scale.
Signals: llm-inference, inference, inference-engine, model-serving, llm-serving, llama-cpp, vllm, triton-inference-server
410 results
PytorchAutoDrive: Segmentation models (ERFNet, ENet, DeepLab, FCN...) and Lane detection models (SCNN, RESA, LSTR, LaneATT, BézierLaneNet...) based on PyTorch with fast training, visualization, benchmarking & deployment help
Aim-bot based on AI for all FPS games
YOLOv5 Series Multi-backbone(TPH-YOLOv5, Ghostnet, ShuffleNetv2, Mobilenetv3Small, EfficientNetLite, PP-LCNet, SwinTransformer YOLO), Module(CBAM, DCN), Pruning (EagleEye, Network Slimming), Quantization (MQBench) and Deployment (TensorRT, ncnn) Compression Tool Box.
Real-time pose estimation accelerated with NVIDIA TensorRT
Stable Diffusion in NCNN with c++, supported txt2img and img2img
🔥🔥🔥TensorRT for YOLOv8、YOLOv8-Pose、YOLOv8-Seg、YOLOv8-Cls、YOLOv7、YOLOv6、YOLOv5、YOLONAS......🚀🚀🚀CUDA IS ALL YOU NEED.🍎🍎🍎
TensorRT MODNet, YOLOv4, YOLOv3, SSD, MTCNN, and GoogLeNet
YOLOv8 using TensorRT accelerate !
YOLOv4, YOLOv4-tiny, YOLOv3, YOLOv3-tiny Implemented in Tensorflow 2.0, Android. Convert YOLO v4 .weights tensorflow, tensorrt and tflite
PyTorch ,ONNX and TensorRT implementation of YOLOv4
Tengine is a lite, high performance, modular inference engine for embedded device
An easy to use PyTorch to TensorRT converter
Implementation of popular deep learning networks with TensorRT network definition API
🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.
DAWNBench: An End-to-End Deep Learning Benchmark and Competition
A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of vLLM).
T-GATE: Temporally Gating Attention to Accelerate Diffusion Model for Free!
optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052
⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.
AICI: Prompts as (Wasm) Programs
🍅🍅🍅YOLOv5-Lite: Evolved from yolov5 and the size of model is only 900+kb (int8) and 1.7M (fp16). Reach 15 FPS on the Raspberry Pi 4B~
This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
Add bisenetv2. My implementation of BiSeNet
多个SVC/TTS的C++推理库
24,535 repositories in the index in total.
