Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
| Date | Stars |
|---|---|
| 2026-07-31 | 1154 |
| 2026-08-05 | 1158 |
| 2026-08-06 | 1158 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# wllama - Wasm binding for llama.cpp
WebAssembly binding for [llama.cpp](https://github.com/ggerganov/llama.cpp)
👉 [Try the demo app](https://huggingface.co/spaces/ngxson/wllama)
👉 See the [blog post](https://reeselevine.github.io/llamas-on-the-web/) introducing WebGPU support in llama.cpp and wllama
📄 [Documentation](https://github.ngxson.com/wllama/docs/)
For changelog, please visit [releases page](https://github.com/ngxson/wllama/releases)
> [!IMPORTANT]
>
> **🔥🔥 V3 is out, with WebGPU, multimodal and tool calling support. Read the [V3 release guide](./guides/intro-v3.md)**
>
> For compatibility issues, please refer to [@wllama/wllama-compat](./compat/README.md)
## Features
- 🔌 OpenAI-compatible API (fully-typed built-in)
- 🚀 WebGPU support
- 🔥 Multimodal support (image and audio file input)
- 🔥 Tool calling support
- Can run inference directly on browser (using [WebAssembly SIMD](https://emscripten.org/docs/porting/simd.html)), no backend or GPU is needed!
- No runtime dependency (see [package.json](./package.json))
- Ability to split the model into smaller files and load them in parallel (same as `split` and `cat`)
- Auto switch between single-thread and multi-thread build based on browser support
- Inference is done inside a worker, does not block UI render
- Pre-built npm package [@wllama/wllama](https://www.npmjs.com/package/@wllama/wllama)
Limitations:
- To enable multi-thread, you must add `Cross-Origin-Embedder-Policy` and `Cross-Origin-Opener-Policy` headers. See [this discussion](https://github.com/ffmpegwasm/ffmpeg.wasm/issues/106#issuecomment-913450724) for more details.
- Max file size is 2GB, due to [size restriction of ArrayBuffer](https://stackoverflow.com/questions/17823225/do-arraybuffers-have-a-maximum-length). If your model is bigger than 2GB, please follow the **Split model** section below.
## Code demo and documentation
Demo:
- Basic usages with completions and embeddings: https://github.ngxson.com/wllama/examples/basic/ ([source code](./examples/basic/index.html))
- Embedding and cosine distance: https://github.ngxson.com/wllama/examples/embeddings/ ([source code](./examples/embeddings/index.html))
- Multimodal (vision) completion: https://github.ngxson.com/wllama/examples/multimodal/ ([source code](./examples/multimodal/index.html))
- Tool calling: https://github.ngxson.com/wllama/examples/tools/ ([source code](./examples/tools/index.html))
## How to use
### Use Wllama inside React Typescript project
Install it:
```bash
npm i @wllama/wllama
```
Then, import the module:
```ts
import { Wllama } from '@wllama/wllama';
let wllamaInstance = new Wllama(WLLAMA_CONFIG_PATHS, ...);
// (the rest is the same with earlier example)
```
For complete code example, see [examples/main/src/utils/wllama.context.tsx](./examples/main/src/utils/wllama.context.tsx)
NOTE: this example only covers completions usage. For embeddings, please see [examples/embeddings/index.html](./examples/embeddings/index.html)
### WebGPU support
WebGPU support is introduced via [PR #215](https://github.com/ngxson/wllama/pull/215).
Upon updating to V3.1, WebGPU will be enabled automatically. By default, all layers will be offloaded to GPU. If the model is too big to fit into VRAM, you can manually adjust the number of layers via the `n_gpu_layers` parameter of `LoadModelParams`. Example:
```js
// (optionally) will allow running WebGPU on Firefox via compat mode; performance will be significantly degraded
wllama.setCompat('default', 'firefox_safari');
await wllama.loadModel(files, {
n_gpu_layers: 4, // meaning 4 layers are offloaded to GPU; set to 0 to disable GPU inference
});
```
### Prepare your model
- It is recommended to split the model into **chunks of maximum 512MB**. This will result in slightly faster download speed (because multiple splits can be downloaded in parallel), and also prevent some out-of-memory issues. **See the "Split model" seExcerpt of 9,086 characters
Read on GitHub
Xuan-Son Nguyen · @huggingface · France
169
Victor Nogueira · @globalnetx · Brazil
7
2
1
1
Reese Levine
1
1
Olivier Cornelis · olicorp · France
1
Thomas Steiner · @google Google
1
Would you bet a product on this? Bounded 0–100 and slow moving.
matched fp:33410d8bc93d3358, topic:llm, topic:llama
matched fp:33410d8bc93d3358, topic:llamacpp