Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Demonstration of running a native LLM on Android device.
| Date | Stars |
|---|---|
| 2026-07-31 | 258 |
| 2026-08-05 | 258 |
| 2026-08-06 | 258 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Native-LLM-for-Android Quick Try: [Qwen3.5-0.8B-Android](https://drive.google.com/file/d/1O7Pz_MWXMTt6sKRMa9wwrmk5k1PmVjPK/view?usp=sharing) ### Qwen3.5-0.8B / 4096 Context  ## Overview Demonstration of running a native Large Language Model (LLM) on Android devices. Currently supported models include: - **Qwen3.5**: 0.8B, 2B, 4B... - **Qwen3**: 0.6B, 1.7B, 4B... - **Qwen3-VL**: 2B, 4B... - **Qwen2.5-VL**: 3B - **MiniCPM-DPO/SFT**: 1B, 2.7B - **Gemma-3-it**: 1B, 4B... - **Phi-4-mini-Instruct**: 3.8B - **Llama-3.2-Instruct**: 1B - **InternVL-Mono**: 2B - **InternLM-3**: 8B - **Seed-X**: [PRO-7B](https://modelscope.cn/models/ByteDance-Seed/Seed-X-PPO-7B), [Instruct-7B](https://modelscope.cn/models/ByteDance-Seed/Seed-X-Instruct-7B) - **HunYuan**: [MT2-1.8B/7B](https://modelscope.cn/models/Tencent-Hunyuan/Hy-MT2-1.8B) ## Update - 2026/05/28：Update Qwen3.5-Dense - 2026/05/25：Update HunYuan-MT-2 - 2026/01/04：Update HunYuan-MT-1.5 - 2025/11/11：Update Qwen3-VL. - 2025/09/07：Update HunYuan-MT. - 2025/08/02：Update Seed-X. - 2025/04/29：Update Qwen3. - 2025/04/05：Update Qwen2.5, InternVL-Mono `q4f32` + `dynamic_axes`. - 2025/02/22：Support loading with low memory mode: `Qwen`, `QwenVL`, `MiniCPM_2B_single`; Set `low_memory_mode = true` in `MainActivity.java`. - 2025/02/07：**DeepSeek-R1-Distill-Qwen**: 1.5B (Please using `Qwen v2.5 Qwen_Export.py`) ## Tokenizer Files - The `tokenizer.cpp` and `tokenizer.hpp` files are sourced from the [mnn-llm repository](https://github.com/alibaba/MNN/tree/master/transformers/llm/engine/src). ## Exporting Models 1. Navigate to the `Export_ONNX` folder. 2. Follow the comments in the Python scripts to set the folder paths. 3. Execute the `***_Export.py` script to export the model. 4. Quantize or optimize the ONNX model manually. ## Quantization Notes - Use `onnxruntime.tools.convert_onnx_models_to_ort` to convert models to `*.ort` format. Note that this process automatically adds `Cast` operators that change FP16 multiplication to FP32. - The quantization methods are detailed in the `Do_Quantize` folder. ## Additional Resources - Explore more projects: [DakeQQ Projects](https://github.com/DakeQQ?tab=repositories) ## Performance Metrics ### Qwen | OS | Device | Backend | Model | Inference (1024 Context) | |:----------:|:------------:|:-----------------------:|:----------------------:|:------------------------:| | Android 13 | Nubia Z50 | 8_Gen2-CPU | Qwen-2-1.5B-Instruct<br>q8f32 | 20 token/s | | Android 15 | Vivo x200 Pro | MediaTek_9400-CPU | Qwen-3-1.7B-Instruct<br>q4f32<br>dynamic | 37 token/s | | Harmony 4 | P40 | Kirin_990_5G-CPU | Qwen-3-1.7B-Instruct<br>q4f32<br>dynamic | 18.5 token/s | | Harmony 4 | P40 | Kirin_990_5G-CPU | Qwen-2.5-1.5B-Instruct<br>q4f32<br>dynamic | 20.5 token/s | | Harmony 4 | P40 | Kirin_990_5G-CPU | Qwen-2-1.5B-Instruct<br>q8f32 | 13 token/s | | Harmony 3 | 荣耀 20S | Kirin_810-CPU | Qwen-2-1.5B-Instruct<br>q8f32 | 7 token/s | ### QwenVL | OS | Device | Backend | Model | Inference (1024 Context) | |:----------:|:------------:|:-----------------------:|:-----------------:|:------------------------:| | Android 13 | Nubia Z50 | 8_Gen2-CPU | QwenVL-2-2B<br>q8f32 | 15 token/s | | Harmony 4 | P40 | Kirin_990_5G-CPU | QwenVL-2-2B<br>q8f32 | 9 token/s | | Harmony 4 | P40 | Kirin_990_5G-CPU | QwenVL-2.5-3B<br>q4f32<br>dynamic | 9 token/s | 惑 ### DeepSeek-R1 | OS | Device | Backend | Model | Inference (1024 Context) | |:----------:|:------------:|:-----------------------:|:----------------------:|:------------------------:| | Android 13 | Nubia Z50 | 8_Gen
Excerpt of 8,663 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:82b13ca6239d3c9f, llm:repository description: 'Demonstration of running a native LLM on Android device.' Language: C++
matched fp:82b13ca6239d3c9f, llm:repository description: 'Demonstration of running a native LLM on Android device.' Language: C++