Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
On-device LLM Inference Powered by X-Bit Quantization
| Date | Stars |
|---|---|
| 2026-07-31 | 316 |
| 2026-08-05 | 316 |
| 2026-08-06 | 316 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
5.0
growth rate 0.00%/day
# picoLLM Inference Engine
[](https://github.com/Picovoice/picollm/releases)
[](https://github.com/Picovoice/picollm/)
[](https://repo1.maven.org/maven2/ai/picovoice/picollm-android/)
[](https://www.npmjs.com/package/@picovoice/picollm-web)<!-- markdown-link-check-disable-line -->
[](https://cocoapods.org/pods/picoLLM-iOS)<!-- markdown-link-check-disable-line -->
[](https://pypi.org/project/picollm/)
[](https://www.nuget.org/packages/PicoLLM/)
Made in Vancouver, Canada by [Picovoice](https://picovoice.ai)
[](https://twitter.com/AiPicovoice)<!-- markdown-link-check-disable-line -->
[](https://www.youtube.com/channel/UCAdi9sTCXLosG1XeqDwLx7w)
picoLLM Inference Engine is a highly accurate and cross-platform SDK optimized for running compressed large language
models. picoLLM Inference Engine is:
- Accurate; picoLLM Compression improves GPTQ by [significant margins](https://picovoice.ai/blog/picollm-towards-optimal-llm-quantization/)
- Private; LLM inference runs 100% locally.
- Cross-Platform
- Linux (x86_64), macOS (arm64, x86_64), and Windows (x86_64, arm64)
- Raspberry Pi (3, 4, 5)
- Android and iOS
- Chrome, Safari, Edge, and Firefox
- Runs on CPU and GPU
- Free for open-weight models
## Table of Contents
- [picoLLM](#picollm-inference-engine)
- [Table of Contents](#table-of-contents)
- [Showcases](#showcases)
- [Raspberry Pi](#raspberry-pi)
- [Android](#android)
- [iOS](#ios)
- [Cross-Browser Local LLM](#cross-browser-local-llm)
- [Llama-3-70B-Instruct on GeForce RTX 4090](#llama-3-70b-instruct-on-geforce-rtx-4090)
- [Local LLM-Powered Voice Assistant on Raspberry Pi](#local-llm-powered-voice-assistant-on-raspberry-pi)
- [Local Llama-3-8B-Instruct Voice Assistant on CPU](#local-llama-3-8b-instruct-voice-assistant-on-cpu)
- [Accuracy](#accuracy)
- [Models](#models)
- [AccessKey](#accesskey)
- [Demos](#demos)
- [Python](#python-demos)
- [.NET](#net-demos)
- [Node.js](#nodejs-demos)
- [Android](#android-demos)
- [iOS](#ios-demos)
- [Web](#web-demos)
- [C](#c-demos)
- [SDKs](#sdks)
- [Python](#python-sdk)
- [.NET](#net-sdk)
- [Node.js](#nodejs-sdk)
- [Android](#android-sdk)
- [iOS](#ios-sdk)
- [Web](#web-sdk)
- [C](#c-sdk)
- [Releases](#releases)
## Showcases
### Raspberry Pi
[](https://www.youtube.com/watch?v=CeKPXZ_8hkI)
### Android
[](https://www.youtube.com/watch?v=XeUMkue-5lI)
### iOS
[](https://www.youtube.com/watch?v=dNK5esdkI0Y)
### Cross-Browser Local LLM
[Live Demo — Works offline!](https://picovoice.ai/picollm/)
### Llama-3-70B-Instruct on GeForce RTX 4090
[](https://www.youtube.com/watch?v=4mcVwbOOIqk)
### Local LLM-Powered Voice Assistant on Raspberry Pi
[![Local LLM-Powered Voice Assistant on Raspberry Pi](https://img.youtube.com/vi/GEndT3RGRvw/Excerpt of 15,964 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:dd4fc2ad7a8da7be, topic:llm, topic:llama, topic:mistral
matched fp:dd4fc2ad7a8da7be, topic:quantization, topic:model-compression, desc:quantization