Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Qwen3.5-122B-A10B on DGX Spark: 28.3 → 51 tok/s (+80%)
| Date | Stars |
|---|---|
| 2026-07-24 | 283 |
| 2026-07-25 | 284 |
| 2026-07-28 | 287 |
| 2026-07-30 | 287 |
| 2026-08-06 | 287 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Qwen3.5-122B-A10B on DGX Spark: 28.3 → 52 tok/s (+82%) [](https://www.apache.org/licenses/LICENSE-2.0) [](.) [](.) [](.) [](https://www.nvidia.com/en-us/products/workstations/dgx-spark/) [](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) [](https://huggingface.co/Intel/Qwen3.5-122B-A10B-int4-AutoRound) [](.) [](.) [](.) [](.) [](https://github.com/vllm-project/vllm) [](.) [](docker/Dockerfile.v2) Optimizations for Qwen3.5-122B-A10B inference on a single NVIDIA DGX Spark from **28.3 to 52 tok/s** (+82%), with 256K context support, no quality degradation. Headline 52 = round of 51.6 tok/s cross-prompt average measured 2026-05-09 on v2.4 (autotune + PR #38325, both default-on); LongCode peak 54.9 tok/s. ## Results | Configuration | tok/s | Improvement | Build | |---|---|---|---| | Baseline (vLLM 0.19 + AutoRound INT4 + FlashInfer) | **28.3** | -- | -- | | + Hybrid INT4+FP8 Dense Layers | **30.8** | +8.8% | step 1 | | + MTP-2 Speculative Decoding | **38.4** | +35.7% | step 2 | | **v2** (+ INT8 LM Head v2) | **51** | **+80%** | **`Dockerfile.v2`** | | **v2.4** (+ `@triton.autotune` on LM Head + vLLM PR #38325 swapAB FP8 SM120, both default-on since 2026-05-09) | **52** (54.9 LongCode peak) | **+82%** | `./install.sh` (default) | | v2-tq (+ TurboQuant KV Cache) | 39 | +38% | `Dockerfile.v2-tq` | The same optimizations also work with Qwen3.5-35B-A3B (same architecture, smaller): **112 tok/s**. > **About the v2.4 numbers** (5 runs × 2 sub-runs = 10 sub-runs, n=50 prompt measurements, `bench_qwen35.sh` 2026-05-09): > > | Prompt | Mean | Median | Std | > |---|---:|---:|---:| > | Q&A 256 | 51.3 | 51.3 | 0.70 | > | Code 512 | 52.8 | 52.9 | 0.23 | > | JSON 1024 | 51.1 | 51.2 | 0.78 | > | Math 64 | 47.8 | 48.1 | 0.81 | > | LongCode 2048 | **54.9** | 55.0 | 0.31 | > > Cross-prompt mean of per-sub-run averages: **51.58 ± 0.30 tok/s**. Headline 52 = round(51.58); LongCode 54.9 reflects the most decode-bound prompt (long sustained generation) where LM Head + shared_expert FP8 paths dominate. > > Composition: wonderwork v2 (canonical 51) + `@triton.autotune` (+1.2% A/B) + vLLM PR #38325 swapAB FP8 SM120 (+0.76% marginal A/B). The +1.1% headline gain (51 → 51.6) is at ~1.5σ of bench noise; the per-prompt deltas (especially LongCode 54.2 → 54.9 and Code 52.0 → 52.8) are above noise individually. See [Optimization 4](#optimization-4-int8-lm-head-v2) for autotune details and [Optimization 5](#optimization-5-vllm-pr-38325-swapab-sm120-fp8-gemm) for PR #38325. ### 256K Context Support v2 supports 256K context out of the box (355K token KV cache). No TurboQuant needed for single-user 256K. | Config | KV Cache | Concurrent Users @ 256K | |---|---|---| | v2 (standard) | 355K tokens | 1 | | v2-tq (TurboQuant) | 1.4M tokens | 5 | --- ## Quick St
Excerpt of 53,026 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:eee59de7b76b71bf, topic:vllm, readme:kv cache, readme:speculative decoding
matched fp:eee59de7b76b71bf, readme:quantization, readme:int8, name:int4
matched fp:eee59de7b76b71bf, topic:cuda