Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Kolosal AI is an OpenSource and Lightweight alternative to LM Studio to run LLMs 100% offline on your device.
| Date | Stars |
|---|---|
| 2026-07-31 | 455 |
| 2026-08-06 | 455 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
## Kolosal AI https://github.com/user-attachments/assets/589cfb48-f806-493d-842b-3b6953b64e79 **Kolosal AI** is an open-source desktop application designed to simplify the training and inference of large language models on your own device. It supports any CPU with **AVX2** instructions and also works with **AMD** and **NVIDIA** GPUs. Built to be lightweight (only ~20 MB compiled), **Kolosal AI** runs smoothly on most edge devices, enabling on-premise or on-edge AI solutions without heavy cloud dependencies. - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) - **Developer:** [Genta Technology](https://genta.tech) - **Community:** [Join our Discord](https://discord.gg/XDmcWqHmJP) ### Key Features 1. **Universal Hardware Support** - AVX2-enabled CPUs - AMD and NVIDIA GPUs 2. **Lightweight & Portable** - Compiled size ~20 MB - Ideal for edge devices like Raspberry Pi or low-power machines 3. **Wide Model Compatibility** - Supports popular models like **Mistral**, **LLaMA**, **Qwen**, and many more - Powered by the [Genta Personal Engine](https://github.com/genta-technology/inference-personal) built on top of [Llama.cpp](https://github.com/ggerganov/llama.cpp), you can see the source code at [https://github.com/genta-technology/inference-personal](https://github.com/genta-technology/inference-personal) 4. **Easy Dataset Generation & Training** - Build custom datasets with minimal overhead - Train models using **UnsLOTH** or other frameworks - Deploy locally or as a server in just a few steps 5. **On-Premise & On-Edge Focus** - Keeps data private on your own infrastructure - Lowers costs by avoiding expensive cloud-based solutions ### Use Cases - **Local AI Inference:** Quickly run LLMs on your personal laptop or desktop for offline or on-premise scenarios. - **Edge Deployment:** Bring large language models to devices with limited resources, ensuring minimal latency and improved privacy. - **Custom Model Training:** Simplify the process of data preparation and model training without relying on cloud hardware. --- ## Credits & Attribution Kolosal AI uses or references the following third-party projects, each licensed under their respective terms: - [Dear ImGui](https://github.com/ocornut/imgui) (MIT License) - [llama.cpp](https://github.com/ggerganov/llama.cpp) (MIT License) - [nativefiledialog-extended](https://github.com/btzy/nativefiledialog-extended) (zlib License) - [nlohmann/json](https://github.com/nlohmann/json) (MIT License) - [stb libraries](https://github.com/nothings/stb) (Public Domain or MIT License) These projects are distributed under their own licenses, separate from Kolosal AI. We are not affiliated with nor endorsed by the above entities. --- ## About Genta Technology We are a small team of students passionate about addressing key concerns in AI such as **energy**, **privacy**, **on-premise**, and **on-edge** computing. Our flagship product is the **Genta Inference Engine**, which allows enterprises to deploy open-source models on their own servers, with **3-4x higher throughput**. This can reduce operational costs by up to **80%**, as a single server optimized by our engine can handle the workload of four standard servers. --- ### Get Involved 1. **Clone the Repository**: [https://github.com/Genta-Technology/Kolosal] 2. **Join the Community**: Ask questions, propose features, and discuss development on our [Discord](https://discord.gg/XDmcWqHmJP). 3. **Contribute**: We welcome pull requests, bug reports, feature requests, and any kind of feedback to improve **Kolosal AI**. --- ## How to Compile 1. [Project Overview](#project-overview) 2. [Directory Structure](#directory-structure) 3. [Prerequisites](#prerequisites) 4. [Cloning and Preparing the Repository](#cloning-and-preparing-the-repository) 5. [Configuring the Project with CMake](#configuring-the-project-with-cmake) 6. [Building the Application
Excerpt of 11,129 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:408349b7c571cdb0, topic:llm, topic:gpt, topic:llama