Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
High-performance In-browser LLM Inference Engine
| Date | Stars |
|---|---|
| 2026-07-31 | 18482 |
| 2026-08-01 | 18484 |
| 2026-08-06 | 18484 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
<div align="center" id="top"> # WebLLM [](https://www.npmjs.com/package/@mlc-ai/web-llm) [](https://chat.webllm.ai/) [](https://discord.gg/9Xpy2HGBuD) [](https://github.com/mlc-ai/web-llm-chat/) [](https://github.com/mlc-ai/mlc-llm/) **High-Performance In-Browser LLM Inference Engine.** [Documentation](https://webllm.mlc.ai/docs/) | [Blogpost](https://blog.mlc.ai/2024/06/13/webllm-a-high-performance-in-browser-llm-inference-engine) | [Paper](https://arxiv.org/abs/2412.15803) | [Examples](examples) </div> ## Overview WebLLM is a high-performance in-browser LLM inference engine that brings language model inference directly onto web browsers with hardware acceleration. Everything runs inside the browser with no server support and is accelerated with WebGPU. WebLLM is **fully compatible with [OpenAI API](https://platform.openai.com/docs/api-reference/chat).** That is, you can use the same OpenAI API on **any open source models** locally, with functionalities including streaming, JSON-mode, function-calling (WIP), etc. We can bring a lot of fun opportunities to build AI assistants for everyone and enable privacy while enjoying GPU acceleration. You can use WebLLM as a base [npm package](https://www.npmjs.com/package/@mlc-ai/web-llm) and build your own web application on top of it by following the examples below. This project is a companion project of [MLC LLM](https://github.com/mlc-ai/mlc-llm), which enables universal deployment of LLM across hardware environments. <div align="center"> **[Check out WebLLM Chat to try it out!](https://chat.webllm.ai/)** </div> ## Key Features - **In-Browser Inference**: WebLLM is a high-performance, in-browser language model inference engine that leverages WebGPU for hardware acceleration, enabling powerful LLM operations directly within web browsers without server-side processing. - [**Full OpenAI API Compatibility**](#full-openai-compatibility): Seamlessly integrate your app with WebLLM using OpenAI API with functionalities such as streaming, JSON-mode, logit-level control, seeding, and more. - **Structured JSON Generation**: WebLLM supports state-of-the-art JSON mode structured generation, implemented in the WebAssembly portion of the model library for optimal performance. Check [WebLLM JSON Playground](https://huggingface.co/spaces/mlc-ai/WebLLM-JSON-Playground) on HuggingFace to try generating JSON output with custom JSON schema. - [**Extensive Model Support**](#built-in-models): WebLLM natively supports a range of models including Llama 3, Phi 3, Gemma, Mistral, Qwen(通义千问), and many others, making it versatile for various AI tasks. For the complete supported model list, check [MLC Models](https://mlc.ai/models). - [**Custom Model Integration**](#custom-models): Easily integrate and deploy custom models in MLC format, allowing you to adapt WebLLM to specific needs and scenarios, enhancing flexibility in model deployment. - **Plug-and-Play Integration**: Easily integrate WebLLM into your projects using package managers like NPM and Yarn, or directly via CDN, complete with comprehensive [examples](./examples/) and a modular design for connecting with UI components. - **Streaming & Real-Time Interactions**: Supports streaming chat completions, allowing real-time output generation which enhances interactive applications like chatbots and virtual assistants. - **Web Worker & Service Worker Support**: Optimize UI performance and manage the lifecycle of models efficiently by offloading computations to separate worker threads o
Excerpt of 25,785 characters
Read on GitHub
Charlie Ruan · UC Berkeley
192
Nestor Qin · NVIDIA · United States
46
Akaash Parthasarathy · Carnegie Mellon University · United States
33
Tianqi Chen · CMU, NVIDIA
28
Ruihang Lai · Carnegie Mellon University · United States
27
Hongyi Jin · cmu · United States
23
Rick Zhou · @linkedin
13
Sudeep Agarwal · United States
7
5
Ikko Eltociear Ashimine · Japan
4
4
Victor Nogueira · @globalnetx · Brazil
3
François Beaufort · @GoogleChrome · France
3
Shushi Hong · Carnegie Mellon University · United States
3
3
2
2
2
1
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:778b90be5d731505, topic:llm, topic:language-model
matched fp:778b90be5d731505, topic:deep-learning
matched fp:778b90be5d731505, topic:chatgpt