Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
A Pure Rust based LLM, VLM, VLA, TTS, OCR Inference Engine, powering by Candle & Rust. Alternate to your llama.cpp but much more simpler and cleaner..
| Date | Stars |
|---|---|
| 2026-07-31 | 430 |
| 2026-08-06 | 430 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Crane 🦩 > Crane focusing on accelerate LLM inference speed with the power of kernels in candle framework, while reducing development overhead, make it portable and fast run model on both CPU and GPU.   **Crane (🦩)** - **C**andle-based **R**ust **A**ccelerated **N**eural **E**ngine A high-performance inference framework leveraging Rust's Candle for maximum speed on CPU/GPU. **Supported Models**: - [ ] more to come.... - [ ] Qwen3-VLA, Qwen3.5-GR00T; - [x] Qwen 3.5 (0.8B; hybrid Gated Delta Net + softmax attention, CPU/CUDA/Metal) + Ornith-1.0-9B (agentic, tool calling) - [x] Hunyuan Dense - [x] Gemma 4 (text and vision; no audio) - [x] Qwen3 VL (2B, 4B) - [x] PaddleOCR VL 0.9B / 1.5 - [x] Qwen3 (0.6B ~ 30B+) - [x] Qwen 2.5 (0.5B ~ 72B) - [x] Moonshine ASR - [x] Silero VAD - [x] 🎙️ Qwen3-TTS (12Hz, 24kHz, 16-codebook RVQGAN + native Candle decoder, voice cloning) - [x] 🎙️ [Voxtral-4B-TTS](https://arxiv.org/abs/2603.25551) (12.5Hz, 24kHz, autoregressive + flow-matching, 20 preset voices across 10 languages) - [ ] 🎙️ TTS: [Spark-TTS](https://github.com/SparkAudio/Spark-TTS) | [Orpheus-TTS](https://github.com/canopyai/Orpheus-TTS) (WIP) submit your models make other users use it easier! **You can run Qwen3-VL 2B with fast speed in local, 50x faster than native PyTorch on M1/M2/M3.** **Key Advantages**: - 🚀 **Blazing-Fast Inference**: Outperforms native PyTorch with Candle's optimized kernels - 🦀 **Rust-Powered**: Eliminate C++ complexity while maintaining native performance - 🍎 **Apple Silicon Optimized**: Achieve GPU acceleration via Metal on macOS devices - 🤖 **Hardware Agnostic**: Unified codebase for CPU/CUDA/Metal execution - 🌐 **OpenAI compatible API**: Supports OpenAI and SGLang interfaces **Crane maybe the fastest (both speed and develop speed) framework you can use to build your AI applications!** Crane using candle as the only dependencies, inference with **fastest** speed cross CPUs and GPUs, while your code can be compiled into binary same as llama.cpp does but much more clean and simpler. **Most important!!!** *Crane is not a low-level SDK, you can call AI abilities out-of-box with ease*. We include: - Basic LLM chat; - VLM chat; - OCR with VLM; - VLA (on the way); - TTS; - ASR; - VAD; - .... (Any AI ability you want power with AI.) ## 🔥 Updates - **`2026.07.03`**: 🗜️ Qwen 3.5 quantization & memory — load community **GGUF** files directly (`--model-path model.gguf`, llama.cpp `qwen35` layout incl. the hybrid GDN blocks, arch auto-detected from the header, tokenizer + chat-template read from GGUF metadata so **no sibling files required**), or quantize a safetensors checkpoint at load time with **`--quant q4k|q8_0|…`** / `CRANE_ISQ` (in-situ quantization via candle `QMatMul`, no conversion step). New **`--dtype f16|bf16|f32`** flag; Qwen 3.5 now defaults to **F16 on Apple Metal**. Qwen3.5-0.8B on Apple Silicon: ~1.2 GB (Q4_0 GGUF) / ~2.0 GB (F16, new default) / ~3.7 GB (old F32 default). - **`2026.06.30`**: 🚀 Qwen 3.5 / Ornith follow-up — K=128 register-resident CUDA recurrence kernel (~5× prefill, ~7.8× recurrence-only on RTX 3090), per-token int8 / int4 K/V cache backends (~2× / ~4× smaller via `CRANE_KV_QUANT`), and Ornith tool-calling support (HF-byte-identical chat template via `AutoTokenizer::apply_chat_template_with_tools`, end-to-end `ornith_tools` example). - **`2026.06.29`**: 🌀 Qwen 3.5 support — hybrid Mamba/Transformer (Gated Delta Net + softmax attention), runs on CPU, NVIDIA CUDA, and Apple Metal. New `crane-core/src/ops/gdn/` module with a fused CUDA recurrence kernel for the linear-attention path. - **`2026.05.04`**: Gemma 4 support added for text and vision models (audio is not supported); - **`2026.02.23`**: 🎙️ Qwen3-TTS support added — full Talker + Code Predictor transformer in Candle, native speech-tokenizer decoder (ONNX fallback), voice cloning (Base model ICL), OpenAI `/v1/audio/speech
Excerpt of 20,019 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1f95cacb27a2ed4d, topic:llama-cpp, desc:inference engine