A complete, practical guide to running large language models locally on AMD Strix Halo / Ryzen AI MAX+ 395 systems with Radeon 8060S (gfx1151) and 96GB/128GB unified memory.
Covers BIOS config, Ubuntu 24.04/kernel setup, Ollama, llama.cpp Vulkan/RADV, ROCm/HIP experiments, vLLM notes, 70B/120B and selected 284B GGUF capacity evidence, benchmarks, raw logs, and reproducibility checks.
AMD now publicly frames Ryzen AI Halo-class systems as a local-AI and developer-platform direction. This repository is the independent practical layer: copyable setup, measured rows, raw evidence, failures, and community reproductions. It is not official AMD or OEM endorsement. See RYZEN_AI_HALO_CONTEXT.md.
Project website: https://strixhaloguide.com/. This GitHub repository remains the source of truth for setup commands, benchmark claims, and raw evidence.
Maintainer credibility is public and reviewable: 15+ merged upstream contributions, including llama.cpp, AMD's Lemonade local-AI server, a Strix Halo detection fix in llmfit, OpenAI's official .NET SDK, and Kubernetes SIG inference-perf. See UPSTREAM_CONTRIBUTIONS.md for every PR link, its scope, and honest boundaries. Upstream acceptance strengthens confidence in the engineering process; it does not replace the raw evidence required for each benchmark claim.
| You are... | Best first page | What it answers |
|---|---|---|
| Looking for the readable project overview | Strix Halo Guide website | The buyer/setup path, evidence model, and routes into the canonical technical source |
| Setting up a machine you already own | Quick Start or the short setup answer | BIOS, Ubuntu, memory, Vulkan/RADV, Ollama, and the first working model |
| Deciding what model or backend to run | Best Known Profiles and Current Models | Easy chat, direct speed, long context, serving, multimodal, capacity, and experimental routes |
| Evaluating Qwen3.8 claims | Qwen3.8 on Strix Halo | Why measured 20.42 t/s and community 22-65 t/s routes are different claims; what is verified locally and what still needs reproduction |
| Comparing or buying Strix Halo hardware | Buyer Use Cases, Buying Guide, and cross-OEM evidence | Memory fit, OS/backend tradeoffs, OEM portability, power, thermals, and missing proof |
| Reproducing or contributing benchmarks | Reproducibility, headline claim index, and benchmark issue | Exact commands, metadata, raw evidence, caveats, and community credit |
| Reviewing the project for a vendor or publication | One-page brief, partnership scope, and disclosure policy | Which buyer uncertainty the evidence removes and how independence is protected |
Current evidence state — September 19, 2026: Qwen3.8 27B is measured through
the official Ollama route. The August 30 b10687 Vulkan sentinel and Flash-Next
scout are now indexed with their own stack and caveats; see
the measured update.
The September 19 availability check
lists newer candidates, including Ollama 0.34.2, without promoting them. The
machine-readable freshness record is data/public_state.json.
What you get:
- A copyable Ubuntu + Vulkan/RADV setup for Ollama and
llama.cpp. - Workload-to-model/backend recommendations instead of one undifferentiated leaderboard.
- Direct, API/server, MTP/speculative, concurrency, capacity, power, NPU, RPC, first-party, and community claims kept separate.
- Headline rows linked to CSVs, raw logs, charts or explicit
n/a, plus preserved failures and corrections. - Cross-OEM evidence covering 13 systems or independent sources from 10 credited community benchmark contributors.
Measured primarily on one Beelink GTR9 Pro. Community results are kept separate from local headline claims. This repository ships docs, scripts, data, and charts only; no
.exe, binary.zip, browser extensions, or model weights. Raw evidence, commands, caveats, and corrections are linked so results can be checked instead of taken on trust.
Quick Start | Qwen3.8 | Profiles | Current Models | Evidence | Community | Reproduce | Contribute
If this guide saves you setup or research time, a GitHub star helps other Strix Halo owners find it; a reproducible result or correction helps even more.
- You own, ordered, review, sell, or are evaluating a Strix Halo / Ryzen AI MAX system for local AI.
- You want a working route and the evidence needed to understand where it transfers—or fails to transfer—to another system.
- You need independent buyer guidance that can coexist with future disclosed vendor or affiliate relationships.
| Need | Source of truth |
|---|---|
| Working beginner setup | Quick Start, setup.sh, and STRIX_HALO_LOCAL_LLM_SETUP.md |
| Current Qwen3.8 decision | QWEN38_STRIX_HALO.md and data/qwen38_route_matrix.csv |
| Known-good workload profiles | BEST_KNOWN_PROFILES.md and data/best_known_profiles.csv |
| Auditable benchmark claims | Headline Evidence, data/headline_claims.csv, and data/raw/ |
| Cross-OEM buyer evidence | SYSTEM_EVIDENCE_MATRIX.md, COMMUNITY_RESULTS.md, and BUYER_PATH_VALIDATION.md |
| Current blockers and test priorities | CURRENT_MODELS.md, ROCM_VLLM_BUGWATCH.md, and data/current_test_queue.csv |
| Question | Current answer |
|---|---|
| Best beginner route | Ollama with Vulkan/RADV. The fully reboot-qualified general baseline remains 0.31.2; Qwen3.8 is separately measured on 0.32.13. Available Ollama 0.34.2 (September 19 check) still needs the controlled upgrade/reboot matrix; the revised installer itself is not yet fresh-install/upgrade qualified. |
| Current Qwen3.8 route | Official Qwen3.8 27B Q4_K_M measured 292.49 prompt t/s and 20.42 generation t/s; image, tools, thinking, and exact retrieval through 50,059 prompt tokens passed. Read the route comparison. |
| Fast direct 30B-class route | Qwen3-Coder 30B-A3B Q4_K_S reached 100.99 tg128 on official b9851; it is a speed-first quant, not the balanced default. |
| Largest direct GGUF tested | DeepSeek V4 Flash 284B UD-IQ2_XXS loaded as a 90.86GB low-bit artifact and measured 13.27 tg128; capacity proof, not broad quality. |
| Experimental server frontier | Repeat-confirmed local MTP/server profiles reach 101-141 t/s, but prompt shape, draft acceptance, runtime, and quant are part of each claim. |
| Cross-system confidence | 13 systems or independent sources and 10 credited community benchmark contributors, with community data separated from first-party Beelink rows. |
| Trust model | Every headline maps to structured/raw evidence or an explicit caveat; failed paths and negative results remain public. |
| Commercial independence | No affiliate links are present as of September 19, 2026. Future affiliate/vendor support must be disclosed and cannot buy ranking, positive conclusions, or removal of accurate negative findings. |
For those who want to get running as fast as possible:
- BIOS: Set UMA Frame Buffer to 512MB if available; if your BIOS minimum is 2GB, leave it at 2GB. Keep IOMMU enabled/default for laptops, suspend, and NPU use. Disabling it is an optional desktop benchmark profile.
- Install Ubuntu 24.04 LTS. X11 is needed only for a desktop tool that requires it, not headless inference.
- Memory profile: The recorded 128GB Beelink profile uses
amdgpu.gttsize=131072 ttm.pages_limit=31457280. Do not copy these limits to 64GB/96GB systems; follow the scoped manual memory section. Addamd_iommu=offonly for the optional desktop benchmark profile after reading Choose the IOMMU policy. - Driver and power policy: Follow the measured Mesa/RADV route and record the active power manager. Preserve an existing policy by default;
tuned accelerator-performanceis an opt-in reproduction profile, not a universal requirement or guaranteed speedup. - Ollama: Install, configure Vulkan backend with
OLLAMA_VULKAN=1,OLLAMA_IGPU_ENABLE=1, andHIP_VISIBLE_DEVICES=-1. WithoutOLLAMA_IGPU_ENABLE=1, measured builds can detect the Radeon 8060S and still fall back to CPU-only inference. - Test:
ollama run qwen3.6:35b-a3b-- the measured Ollama 0.31.2 system-service path reached about 60 t/s generation. Exact speed depends on runtime, model, power state, and background load.
Use the setup script below for the automated path. The phases later in this README are the manual reference and fallback path if you want to inspect or reproduce each change yourself.
Want the current dense multimodal model instead? Read the
Qwen3.8 27B route decision before changing the
reboot-qualified default: the official model is measured here, but its runtime,
context boundary, and community performance routes need different caveats.
Existing-user result, September 19: the scoped qualification passed text, separate-model image, executed tools and pinned WebUI on the existing Ollama 0.32.15 service after restart. The historical fresh-install reboot pin stays 0.31.2; isolated 0.34.2 is useful but not default. These tests are not a new clean-install or full-host-reboot qualification.
Scope before running: this automatic route is for 128GB-class systems with at least 120GiB visible RAM. It is not a general 64GB/96GB installer. Configuration handling has offline fixture coverage; the revised script's fresh-install and upgrade paths still need hardware qualification. Review the script and existing configuration first: a later conflict can stop a partially completed run; this is not a transactional installer with a complete automatic rollback.
Fresh installations request Ollama 0.31.2, the reboot-qualified runtime.
An existing installation is retained, with its version reported; it is not
automatically downgraded. Set OLLAMA_VERSION explicitly to test another
release. This pins only the requested runtime: model tags, Mesa and other
packages can change, so record the actual stack and complete the reboot/GPU
checks before comparing with a measured profile. See
Ollama version selection.
If you've already set your BIOS (UMA = 512MB if available, or 2GB if that is your vendor minimum; leave IOMMU enabled/default unless you deliberately choose the desktop benchmark profile) and installed Ubuntu 24.04:
git clone https://github.com/hogeheer499-commits/strix-halo-guide
cd strix-halo-guide
bash setup.shInspect the script first with less setup.sh before running it: it edits GRUB boot parameters, adds the kisak Mesa PPA, and runs a full apt upgrade.
For unattended copy/paste installs, the same script can also be run as:
curl -fsSL https://raw.githubusercontent.com/hogeheer499-commits/strix-halo-guide/main/setup.sh | bashThis installs the Linux-side Vulkan/RADV + Ollama path, configures Ollama for Vulkan, pulls a model, and prepares a verification benchmark. If the script changes boot parameters, reboot first and then run bash ~/bench-ollama.sh.
This is the quick "what can I actually run on my AI PC?" view. It is not the full benchmark list; see What You Can Run for more models and Headline Evidence for the audit trail.
| What you want to do | Measured local result | Practical takeaway | Evidence |
|---|---|---|---|
| Fastest direct 30B-class Qwen MoE row | Qwen3-30B-A3B-Instruct-2507 IQ4_XS: 100.04 t/s direct llama.cpp Vulkan/RADV on b9467; b9544 control measured 103.18 tg128 r10 | First local direct llama-bench row above 100 t/s. Treat it as a separate general-instruct Qwen route, not as a Qwen3-Coder replacement or balanced-default claim. |
headline claims, raw r50, b9544 control |
| Fastest current small-MoE scout | LFM2.5 8B-A1B Q4_K_M: 168.96 tg128 in pp512/tg128, 170.02 t/s generation-only, b9544 control at 176.48 tg128 r10 | Shows how fast newer small active-parameter MoE routes can be on Strix Halo. Do not compare it as a 30B-class coding/reasoning replacement. | headline claims, raw latest/int-dot, b9544 control |
| Fastest local coding speed | Qwen3-Coder 30B-A3B Q4_K_S: 100.99 t/s direct llama.cpp Vulkan/RADV on the official b9851 release binary; older strict-clean b9179 row measured 98.51 t/s | Speed-first quant candidate. Use it when raw t/s matters and you accept the quality tradeoff. This is direct llama-bench, not MTP/server speculation. |
headline claims, b9851 raw r50, older strict-clean raw r50 |
| Fast balanced local coding model | Qwen3-Coder 30B-A3B UD-Q4_K_XL: 96.76 t/s direct llama.cpp Vulkan/RADV on current b9049 | Strong first model for coding scripts, editors, and agent loops. | headline claims, raw run |
| Newer Qwen coding model | Qwen3-Coder-Next 80B-A3B IQ4_XS: 61.91 t/s direct llama.cpp Vulkan/RADV on b9467 | Modern coding-model row for people who want current Qwen Coder-Next rather than the older 30B speed headline. Use it for capability/currentness, not maximum raw t/s. | benchmarks CSV, raw run |
| Easy private chat setup | Qwen3.6 35B-A3B Q4_K_M: 60.57 t/s warm API generation through the normal Ollama 0.31.2 system service with OLLAMA_IGPU_ENABLE=1; vision, service restart, and full-host reboot persistence passed |
This is the copyable default for model pulling, Open WebUI, vision, and simple local chat. A later controlled local-binary comparison measured 0.31.1/0.31.2/0.32.0 at 72.55/73.19/73.20 t/s, so do not treat the earlier 60.57-versus-71.82 gap as a version-wide regression. | headline claims, raw service run, raw controlled comparison |
| Fine-tune, export, and reload a local model | Pinned ROCm 7.2 Unsloth route: Radeon GPU gate, one SFT step, checkpoint inference, Q4_K_M GGUF export, ROCm llama.cpp inference, and post-restart artifact load all passed |
End-to-end developer workflow evidence on a retail box. The Qwen3 0.6B one-step run is a plumbing smoke, not a quality or performance headline. | Unsloth guide, raw evidence |
| Fast all-rounder direct path | Qwen3.6 35B-A3B UD-Q4_K_M: 62.56 t/s direct llama.cpp Vulkan/RADV on current b9049 | Use this when you care more about speed and control than the easiest UI. | headline claims, raw run |
| Fastest Qwen3.6 direct path | Qwen3.6 35B-A3B Q4_0: 81.30 t/s direct llama.cpp Vulkan/RADV on current b9049 | Speed-first option. Use the default/balanced quant if quality matters more than raw t/s. | max campaign, raw run |
| Qwen3.6 27B dense control | Official Qwen3.6 27B MTP Q8_0: 7.61-7.74 t/s without MTP, 14.59-14.69 t/s with the best MTP setting; a direct b9467 llama-bench follow-up measured 7.70 t/s tg128 |
Useful practical row for people comparing 27B versus 35B-A3B. It runs, but this dense Q8 route is much slower than the 35B-A3B MoE paths and is not a speed candidate. | Performance notes, MTP CSV, raw b9235, raw latest, raw b9467 |
| Experimental Qwen3.6 MTP server path | Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn: 101.16 t/s best local Beelink six-prompt average on b9360, with t16 repeats at 101.15 / 101.10 / 101.06 t/s; first GMKtec community reproduction reached 93.29 t/s on b9235 | Advanced speculative-decoding route. Useful if you are testing a local API server; keep separate from direct llama-bench. |
MTP notes, MTP CSV, local raw, GMKtec raw |
| Experimental Gemma 4 QAT MTP server path | Gemma 4 26B-A4B QAT UD-Q4_K_XL + matched Q4_0 MTP head: 110.00 t/s best repeat, 107.42 t/s T3-only repeat, 102.69 t/s cold repeat, 73.96 t/s no-spec baseline on ac4cddeb0 | Strong current-model server route: matched QAT MTP heads can materially lift a current Google model on Strix Halo. Treat as llama-server speculative evidence, not a direct llama-bench replacement. |
headline claims, MTP CSV, warm raw, cold raw, T3-only raw |
| Official Gemma 4 31B QAT multimodal route | Official Q4_0 GGUF: 308.28 pp512 / 11.38 tg128 direct on b10066; narrow text, STRIX 395 vision, and native calculator tool-call smokes passed |
Current dense Google-model compatibility route for buyers who value text, vision, and tools over maximum decode speed. The matched Q8_0 DFlash sidecar loaded but was slower on the measured 5.5K/21.9K synthetic prompt shapes because acceptance stayed low. | benchmarks CSV, MTP CSV, raw evidence |
| Experimental CHADROCK ROCmFP4 MTP server path | CHADROCK ACE/SABER 35B ROCmFP4 through ciru-ai/ROCmFPX: 141.37 t/s mean across three repeats on the exact 3946-token reference profile, with 100% mean draft acceptance |
Fastest repeat-confirmed server/speculative reference profile in the guide, but not a universal speed. Separate 1K/8K/16K profiles measured 78.00/83.85/107.23 t/s as acceptance changed. Treat as advanced ROCmFPX/CHADROCK evidence, not a direct llama-bench replacement. |
ROCMFP4_CHADROCK.md, raw stability profile, MTP CSV |
| Frontier-size agent server | Step 3.7 Flash 198B-A11B ROCmFPX Q3 plus Q8 MTP draft: 23.84 t/s matched 4K no-spec baseline, 34.50 t/s 4K MTP, 33.83 t/s 16K MTP, native tool-call pass, and 256K allocation | Shows the capacity and agent value of 128GB unified memory rather than chasing the fastest small-model row. Advanced pinned runtime; server/MTP result, and the 48K row has one repeat. | ROCmFPX guide, raw evidence, MTP CSV |
| 80B MoE coding/reasoning experiments | Qwen3-Next 80B-A3B UD-Q4_K_XL: 59.06 t/s direct llama.cpp Vulkan/RADV on b9172 | Historical May 80B Qwen-family path measured here; use when model size and 256K context matter more than smallest footprint. | headline claims, raw r20 |
| Open-weight 120B reasoning model | gpt-oss-120b MXFP4: 55.57 t/s direct llama.cpp Vulkan/RADV on current b9049 | 128GB unified memory can run a 117B-parameter MoE locally; this is speed evidence, not a model-quality eval. | headline claims, raw run |
| Current 120B-class GGUF capacity route | Nemotron 3 Super 120B-A12B UD-IQ4_XS: 18.43 t/s direct llama.cpp Vulkan/RADV, b9544 control at 18.93 tg128 r3 | Answers a different buyer question: yes, a current 120B-class MoE GGUF route can run directly on one 128GB Strix Halo box. | headline claims, raw latest/int-dot, b9544 control |
| Current NVIDIA Omni/FP4 route | The same Nemotron 3 Nano Omni MXFP4 artifact improved from 56.56 tg128 on b9747 to 64.26 on b10034; a separate NVFP4 + F16 projector route measured 53.21 tg128 and correctly read STRIX 395 from an image |
Shows both current runtime maintenance and a first-party image-capable Nemotron route. The OCR check is not broad vision/audio/video validation and neither row replaces the Qwen speed headlines. | benchmarks CSV, raw MXFP4 sentinel, raw multimodal scout |
| Current agent/reasoning scouts | Nemotron Cascade 2 30B-A3B IQ4_XS: 78.95 tg128; Qwen AgentWorld 35B-A3B UD-IQ4_XS: rate withheld pending retained-run provenance with a correct terminal-world smoke and 128K allocation pass |
These answer current-model and agent-use-case questions without pretending every new model is a speed headline. Cascade's forced no-think prefix did not hide reasoning; AgentWorld's 128K result is an allocation smoke, not a filled-context quality claim. | CURRENT_MODELS.md, Cascade evidence, AgentWorld evidence Curator: exact retained-run disagreement; rate provenance pending. |
| Local API for tools or several clients | Qwen3-Coder 30B-A3B b9979: 228.18 aggregate t/s stock at np8; opt-in density+dense16 reached 234.12 at np9, while density alone reached 266.07 at np16 | A software dispatch cliff, not memory capacity, can limit multi-user value. Keep stock for low concurrency; advanced users should compare density Vulkan and ROCm at their exact target. | MOE_CONCURRENCY.md, summary CSV, 30B chart |
| FP16 vLLM at 8-16 concurrent requests | Official ROCm 7.14 image, PyTorch 2.11, Qwen3-0.6B: TORCH_BLAS_PREFER_HIPBLASLT=1 improved aggregate throughput by 40.50% / 38.96% / 41.54% at concurrency 8/9/16 |
This reproduces AMD's Ryzen AI batch-8+ workaround without changing the host. It is a small-model FP16 server A/B, not a direct GGUF or 27B/35B claim; concurrency 4 was slightly slower. | ROCm/vLLM notes, processed A/B, raw evidence |
Historical ROCm/HIP llama.cpp allocation smoke |
Official b10046, Qwen3-0.6B Q8_0: 4666.05 pp512 / 208.73 tg128; full 120,124 MiB free UMA detected and ROCm_Host buffers allocated |
Reproduced full-UMA discovery without HSA_OVERRIDE_GFX_VERSION, but open issue #26209 and the model-scoped v0.4.1 mitigation controls do not make the tiny smoke qualify long-context, multimodal, multi-slot, or practical-model correctness. |
ROCm/HIP notes, raw evidence |
| Long documents or codebase context | Qwen3.6 35B-A3B: 32.23 t/s decode after a filled 128K KV cache | Long-context use is possible, but prompt ingestion cost matters. | filled KV CSV, chart |
| Large-model proof point | MiniMax M2.7 230B-class MoE loaded and generated locally; Llama 4 Scout 109B measured 18.32 t/s historically | 128GB unified memory makes very large local models practical on one compact PC, but capacity and speed are different wins. | CURRENT_MODELS.md, benchmarks CSV |
| Goal | Start with | Why | Evidence |
|---|---|---|---|
| Easiest private local chat | Ollama 0.31.2 system service with Vulkan/RADV | normal install path; model pulling, vision, restart/reboot persistence, and Open WebUI compatibility | 60.57 t/s fully qualified service run; controlled isolated 0.31.1/0.31.2/0.32.0 binaries later measured 72.55-73.20 t/s, data/benchmarks.csv |
| Fast coding or scripts on one machine | llama-server Vulkan/RADV |
fastest measured Qwen3.6 path at 1-4 parallel requests | SERVER_SHOOTOUT.md |
| Speculative decoding experiments | llama-server MTP on current master / ROCmFPX |
measured server speedups on Qwen3.6 MTP GGUFs, Gemma 4 QAT matched MTP heads, and CHADROCK ROCmFP4; Qwen3.6 reached about 101.1 t/s on b9360, Gemma 4 26B-A4B QAT reached 110.0 t/s best-repeat, and the exact CHADROCK reference profile averaged 141.37 t/s over three repeats at 100% draft acceptance | MTP_SPECULATIVE_DECODING.md, ROCMFP4_CHADROCK.md, data/mtp_speculative.csv |
| Several local tools or users hitting one API | Start with stock Vulkan up to 8; above that, compare opt-in density Vulkan and Lemonade ROCm on the exact model | b9979 repeats show density recovers the Vulkan np9 cliff, but the backend winner differs between the tested 30B and 80B models | MOE_CONCURRENCY.md, data/moe_density_gate_summary.csv |
| Long local documents or codebase context | llama-server Vulkan/RADV first, test ROCm/HIP for prompt-heavy ingestion |
128K prompt plus generation completed; HIP can win prompt processing | data/filled_kv_decode.csv, BACKEND_CROSSOVER.md |
| Prompt-heavy ROCm experiments | Keep a reproducible ROCm/HIP + ZenDNN path available, but do not treat it as a decode-speed headline. | A community Beelink GTR9 Pro on CachyOS / kernel 7.0.11 / ROCm 7.2.4 / ZenDNN measured Qwen3.6 27B MTP UD-Q6_K_XL at 303.20 pp5000 on ROCm versus 155.89 pp5000 on Vulkan, while decode stayed around 8 t/s on both backends. |
BACKEND_CROSSOVER.md, COMMUNITY_RESULTS.md |
| NPU sidecar work | Treat the NPU as a possible low-overhead sidecar, not as a replacement for the main iGPU LLM path. | A ciru-ai GMKtec EVO-X2 NixOS artifact kept IOMMU enabled and measured only +3.29% main 64k iGPU workload latency with concurrent NPU load, versus +68.96% with a comparable iGPU auxiliary load. The same package reports FastFlowLM-NPU LFM2.5 1.2B at 32k around 1646 prompt tok/s, 38.18 decode tok/s, and about 2.09GiB RSS. | COMMUNITY_RESULTS.md#gmktec-evo-x2-nixos--npu--rocmfp4-evidence-package, data/community_ciru_evox2_metrics.csv |
| vLLM-style serving experiments | Start with isolated ROCm containers; for ROCm 7.14 plus PyTorch before 2.14, A/B TORCH_BLAS_PREFER_HIPBLASLT=1 at batch 8+ |
The local Qwen3-0.6B FP16 A/B measured about 39-42% more aggregate throughput at concurrency 8/9/16. This validates the setting and container path, but no practical 27B/35B vLLM throughput claim exists yet. | processed A/B, ROCM_VLLM_BUGWATCH.md |
| Sustained inference on a Corsair/Sixunited AXB35 system | Verify the expected EC/fan module and dependent services after every kernel update before applying a clock cap. | A strict three-system campaign found a useful 2400 MHz fleet tradeoff, but historical journals also showed missing ec_su_axb35 modules and failed fan services after updates. The root cause remains unresolved and the cap is not universal. |
THERMAL_STABILITY.md, data/community_thermal_sclk.csv |
These are the practical decisions extracted from the primary Beelink runs plus community reports from Corsair, GMKtec, MS-S1-Max, Nimo, a second Beelink owner stack, and a ciru-ai GMKtec EVO-X2 NixOS/NPU artifact. Use them to avoid retesting dead ends first; follow the evidence links if your setup differs.
| Situation | Do this first | Why | Evidence |
|---|---|---|---|
| One Strix Halo AI PC | Use Vulkan/RADV for GGUF chat, coding, and generation-heavy inference. | It is the fastest measured practical path for the main Qwen MoE rows. | headline claims, COMMUNITY_RESULTS.md |
| Native Linux on another Strix Halo vendor | Expect the same performance class if backend, model, quant, and command match. | GMKtec EVO-X2 96GB on Ubuntu 26.04, Mesa RADV 26.0.3, and llama.cpp b9156 reproduced the guide's Qwen3.6 UD-Q4_K_M row within -0.8% pp512 and -1.7% tg128. | COMMUNITY_RESULTS.md, #16 |
| Comparing Qwen3-Coder rows | Preserve the exact model hash, build and command flags before calling one system faster. | The August 26 GMKtec follow-up supplies hashes and direct rows of 99.08 tg128 for Q4_K_S and 96.75 for UD-Q4_K_XL. It strengthens portability evidence; runtime, model and batch differences still prevent an OEM ranking. |
Community follow-up, raw bundle |
| Handling "older model" criticism | Add current-model rows, but keep speed and capability separate. | Qwen3-Coder-Next IQ4_XS runs locally at 61.91 t/s on the Beelink with b9467. That is a useful modern coding-model result, but it does not replace the older Qwen3-Coder 30B Q4_K_S speed-first headline. | raw Qwen3-Coder-Next run, data/benchmarks.csv |
Tuning llama.cpp batch flags |
Check that -ub is not larger than -b before any other tuning. |
-ub above -b is silently clamped to min(n_batch, n_ubatch) with no warning. A community GTR9 Pro report measured +28.9% pp512 from correcting -b 256 -ub 1024 to -b 2048 -ub 512, and a further +16.9% pp512 / +3.1% tg128 from upgrading stock Mesa 25.2.8 to kisak 26.1.7 (no reboot needed; RADV is userspace). |
COMMUNITY_RESULTS.md, raw note |
| Seeing a direct 100 t/s 30B-class Qwen result | Check the exact model, quant, build, and claim category before comparing. | There are now two separate first-party direct 100-class Qwen rows: Qwen3-Coder 30B-A3B Q4_K_S at 100.99 t/s on official b9851, and Qwen3-30B-A3B-Instruct-2507 IQ4_XS at 100.04 t/s on b9467 with a b9544 control at 103.18 t/s. The Qwen3-Coder row is speed-first, not the balanced default; the 2507 row is a separate general-instruct model. |
b9851 Qwen3-Coder raw, 2507 raw scout, headline claims |
| Seeing Qwen3-Coder around 100 t/s on another Strix Halo box | Treat it as a tuned-system clue, not a default claim. Capture thermals, power policy, Vulkan device line, glslc --version, driver/toolchain details, and exact command. |
A Reddit GMKtec EVO-X2 report saw most Q4_K_S -p 0 -n 128 runs around 99.90 t/s and a best 100.0 t/s after repasting, reseating memory pads, and using GPU high plus CPU EPP performance. Local Beelink b9467 follow-ups stayed around 95.27-96.72 t/s, so thermals/power/toolchain still need to be separated before calling this generally reproducible. |
COMMUNITY_RESULTS.md#reddit-gmktec-evo-x2-tuned-100-ts-report, PERFORMANCE_NOTES.md#vulkan-integer-dot-and-100-ts-reproduction-status, raw reproduction |
| Starting on Windows | LM Studio Vulkan is now a documented Windows path, but keep it separate from Linux llama-bench. |
The first Windows MS-S1-Max report measured a 89.49 tok/s script average through LM Studio with n_parallel=4 and 262K context; the long 512-token prompt rows were around 69-70 tok/s. This is useful Windows buyer evidence, not a same-machine Windows-vs-Linux comparison. |
COMMUNITY_RESULTS.md#windows-lm-studio-ms-s1-max-report, raw Windows report |
| Evaluating a compact non-Beelink chassis | Look for setup metadata, thermal context, and large-model feasibility, not only headline t/s. | The Nimo AI Mini PC issue #4 bundle adds Ubuntu 25.04 / Mesa 25.2.8 / ROCm rows, Qwen 122B-class serving, StepFun 198B-class serving, Qwen3-Coder-Next server rows, DFlash negative/control evidence, Gemma 4 QAT/MTP assistant-head follow-up data, and supplemental fan/power/temperature telemetry. | COMMUNITY_NIMO.md, data/community_nimo_issue4.csv, raw Nimo bundle, Gemma QAT follow-up |
| Choosing an IOMMU policy | Keep IOMMU enabled/default for normal systems, NPU work, and mobile suspend. Use amd_iommu=off only for the optional always-on desktop benchmark profile. |
The Beelink headline profile preserves the measured performance route, while the safer general default avoids silently losing the NPU or deep laptop/tablet sleep. ciru-ai's GMKtec EVO-X2 NixOS artifact also shows an IOMMU-on stack where the NPU can run sidecar work with low measured main-workload impact. | setup policy, COMMUNITY_RESULTS.md#gmktec-evo-x2-nixos--npu--rocmfp4-evidence-package |
| Testing MTP/speculative decoding | Treat MTP as an advanced server route, not a direct benchmark replacement. | The Qwen3.6 MTP IQ4_XS-Q8nextn route has a local b9360 rerun at 101.16 t/s and a GMKtec b9235 reproduction at 93.29 t/s. The Gemma 4 26B-A4B QAT route adds a current Google-model matched-head example at 102.69-110.00 t/s. The exact CHADROCK ACE/SABER reference profile averaged 141.37 t/s over three repeats at 100% acceptance, while lower-acceptance shapes were much slower. | MTP_SPECULATIVE_DECODING.md, ROCMFP4_CHADROCK.md, data/mtp_speculative.csv, #18 |
| The model fits on one Strix Halo box | Do not use llama.cpp RPC for raw single-stream speed. |
2-node RPC lost about 14-22% tg128 on fits-on-one models; 3-node was slower again. | COMMUNITY_RPC.md, data/community_rpc.csv |
| A huge GGUF does not fit on one box | Try ROCm RPC first, starting with the smallest node count that fits. | In the tested MiniMax-M2.7 140.8GB case, one box failed, 2-node ROCm worked, and 3-node ROCm was slower. This is a capacity rule from that case, not a universal speedup rule. | COMMUNITY_RPC.md |
| Testing huge MoE capacity on one box | Separate "loads and processes prompts" from "fast generation". | A Corsair AI Workstation 300 ai-2 community row loaded MiMo-V2.5 UD-IQ2_M, a 310B-total / 15B-active GGUF route, and measured 30.65 pp512 with wall-power/GPU telemetry. The pasted CSV row has n_gen=0, so it is capacity/telemetry evidence, not a tg128 speed claim. |
COMMUNITY_RESULTS.md#corsair-ai-workstation-300-mimo-v25-capacity-row, #26 |
| Building a USB4 Strix Halo cluster | Start with MTU 9000 and pm_qos_resume_latency_us=100. |
MTU 9000 beat 1500 and 65520; pm_qos added about +2% tg128 with only about 1.5 W idle cost per toggled box in the community report. |
USB4_CLUSTER_TUNING.md |
| Choosing Qwen3.6 quantization | Use Q4_K_M/UD-Q4_K_M for balanced defaults; use Q4_0 only when speed matters more than quality. | Q4_0 was faster locally and in the Corsair report, but this guide has not made a model-quality claim for it. | COMMUNITY_RESULTS.md, data/community_results.csv |
| Serving an interactive local API | Prefer single-box llama-server when the model fits. |
Community llama-server TTFT was about 201 ms on 1 node versus about 301 ms on 2-node RPC, with higher RPC variance. |
COMMUNITY_RPC.md, data/community_rpc_server.csv |
| Comparing your t/s to this guide | Treat about 2% tg128 spread as normal between well-matched Strix Halo systems. | Three matched Corsair boxes showed 0.11% pp512 spread and 2.05% tg128 spread; the GMKtec native Qwen3.6 row landed within 2% of the Beelink row. | COMMUNITY_RESULTS.md |
| Estimating electricity and heat | Use wall-power rows as workload-specific context, not universal TDP. | Community wall-power data measured about 150 W / 1.6 J/token for Qwen3-Coder, 148 W / 2.0 J/token for Qwen3.6, 174 W / 3.1 J/token for gpt-oss-120b, and 137 W / 3.4 J/token for Qwen3-Coder-Next. | COMMUNITY_RESULTS.md#whole-system-power, data/community_power.csv |
| Seeing a Vulkan/RADV failure on a huge MoE | Check for per-buffer allocation limits, not only total memory. | MiniMax-M2.7 hit the same 830472192-byte RADV allocation failure on 1-node and RPC follower paths. | data/community_rpc_failures.csv, COMMUNITY_RPC.md |
Have a Strix Halo / Ryzen AI MAX system? Please share results, even if they are slower, failed, or contradict this guide.
- Open a benchmark report with your system, BIOS UMA setting, kernel, Mesa/ROCm versions, backend, model, command, CSV/raw output, and notes.
- Open a power / efficiency report if you can measure wall power, UPS power, or validated board power during the same benchmark command.
- Open a model request if there is a model/backend combination that should be tested.
- Use Discussions for setup questions, comparisons, and early results that are not ready for a benchmark issue yet.
- Power telemetry is useful if you have reliable wall-power data; it helps turn raw tokens/sec into tokens-per-watt context for buying, cooling, and always-on server decisions.
- Current community reports live in
COMMUNITY_RESULTS.md,COMMUNITY_NIMO.md,COMMUNITY_RPC.md,USB4_CLUSTER_TUNING.md, and thecommunity_*CSVs listed indata/README.md. - See
SHARE.mdfor short Reddit/HN/forum text and the current social preview image if you want to share the guide.
This guide is primarily a technical resource for AMD Strix Halo local-AI users. For vendors, reviewers, and partners, it also documents a reproducible way to reduce buyer setup friction and validate real-world local-AI use cases without weakening independent benchmark discipline.
Start with the public vendor and reviewer overview, then inspect the linked GitHub evidence, raw artifacts, negative results, disclosure policy, and upstream contribution record here.
Start with ONE_PAGE_BRIEF.md and PARTNERSHIP.md. Supporting docs cover BEELINK_OUTREACH.md, VENDOR_OUTREACH_PLAN.md, SPONSORSHIP.md, VENDOR_DISCLOSURE.md, BUYER_USE_CASES.md, SPONSOR_ROADMAP.md, TRACTION.md, and OUTREACH_TEMPLATES.md.
No affiliate links are present as of September 19, 2026. If monetized product
links are added later, each relevant link will be labeled and recorded in the
public affiliate link registry. Affiliate
commission does not determine benchmark conclusions, product inclusion, or
ranking.
Best current AMD Strix Halo / Ryzen AI MAX+ 395 local LLM setup from this guide's measured runs. This section is the practical path; the detailed evidence is linked below.
If you are new, do this first:
- Use Ubuntu 24.04.
- Set BIOS UMA Frame Buffer Size to 512MB if available, or 2GB if that is your vendor minimum.
- Keep IOMMU enabled/default if you use suspend, the NPU, RDMA, VFIO, passthrough, or clustering. On an always-on desktop benchmark box,
amd_iommu=offremains an optional measured performance profile. - Use the setup script to install the Vulkan/RADV + Ollama path.
- Start with Ollama for chat, then add Open WebUI if you want a browser UI.
- Move to direct
llama.cpponly when you want exact benchmark control or the fastest measured single-box path.
The quickest sanity check after the setup script finishes is:
ollama run qwen3.6:35b-a3bExpect roughly the same performance class as the guide's Ollama Vulkan/RADV rows if your BIOS, kernel parameters, Vulkan ICD, model, quant, and power profile match.
Choose the backend by what you are trying to do:
| Use case | Do this first | Why |
|---|---|---|
| You want private chat working today | Use the setup script, then run ollama run qwen3.6:35b-a3b. |
Easiest path to model pulling, local chat, and Open WebUI. |
| You want to reproduce the headline speed rows | Use Reproduce One Headline Result. | Exact model, quant, build, and command matter for benchmark comparisons. |
| You want a local API server or MTP tests | Read MTP/speculative decoding and use llama-server. |
Supports serving, batching, long-context tests, and speculative decoding. |
| You have many parallel local requests | Read SERVER_SHOOTOUT.md for the measured Lemonade llamacpp-rocm b1259 profile before planning a new comparison. |
The 8-16 parallel advice was measured on 2026-05-05. Lemonade v11.9.0, checked September 19, is available but unqualified here and has not inherited that result. |
| You are testing prompt-heavy, vLLM, or future server paths | Read BACKEND_CROSSOVER.md and VLLM_BASELINE.md. | Useful for prompt processing, batching, vLLM, and long-context experiments. |
If you only want a working local AI PC, stop after Ollama works. If you want to compare numbers, use the exact commands and evidence links in Reproduce One Headline Result, the AI/search setup summary, headline claim index, and raw evidence under data/raw/.
The machine-readable index for these rows is data/headline_claims.csv.
Dates below are measurement dates. A row being from May does not mean it is stale; it means later checks did not produce a stronger replacement headline. The latest controls and current-model rows are documented in CURRENT_MODELS.md, BENCHMARKS.md, PERFORMANCE_NOTES.md, and the raw evidence links below.
| Claim | Date | Backend | Model | Result | CSV | Raw | Chart | Notes |
|---|---|---|---|---|---|---|---|---|
| Fastest direct 30B-class Qwen MoE route | 2026-06-02; b9544 control 2026-06-07 | llama.cpp Vulkan/RADV b9467; b9544 | Qwen3-30B-A3B-Instruct-2507 IQ4_XS | 100.04 tg128 r50, 1416.03 pp512; b9544 control 103.18 tg128 r10 | benchmarks |
raw r50, b9544 control |
n/a | first local direct llama-bench row above 100 t/s; separate general-instruct Qwen route; not the Qwen3-Coder headline or balanced default |
| Fastest current small-MoE scout | 2026-06-05; b9544 control 2026-06-07 | llama.cpp Vulkan/RADV 2016bf2; b9544 | LFM2.5 8B-A1B Q4_K_M | 168.96 tg128, 3414.61 pp512; generation-only 170.02 tg128; b9544 control 176.48 tg128 r10 | benchmarks |
raw latest/int-dot, b9544 control |
n/a | small active-parameter MoE speed/currentness row; not a 30B-class model replacement |
| Largest current direct GGUF capacity route | 2026-07-16 | llama.cpp Vulkan/RADV b10034 | DeepSeek V4 Flash 284B UD-IQ2_XXS | 155.64 pp512 / 13.27 tg128 r3; deterministic answer 9 |
benchmarks |
raw direct pass |
n/a | pinned 90.86GB ordinary GGUF; direct load/basic-correctness capacity proof, not a speed or broad quality recommendation |
| Current 120B-class direct GGUF route | 2026-06-05; b9544 control 2026-06-07 | llama.cpp Vulkan/RADV 2016bf2; b9544 | Nemotron 3 Super 120B-A12B UD-IQ4_XS | 18.43 tg128, 294.99 pp512; b9544 control 18.93 tg128 r3 | benchmarks |
raw latest/int-dot, b9544 control |
n/a | direct 120B-class MoE route on one 128GB Strix Halo system; capacity/current-model proof, not a speed headline |
| Fastest measured short-context coding MoE speed-first quant | 2026-06-30 | llama.cpp Vulkan/RADV b9851 official release binary | Qwen3-Coder 30B-A3B Q4_K_S | 100.99 tg128 r50, 1423.05 pp512 | benchmarks |
raw r50 |
n/a | speed-first lower-quality quant; not the balanced UD default; older b9179 strict-clean row remains preserved at 98.51 t/s |
| Fast balanced short-context coding MoE | 2026-05-07 | llama.cpp Vulkan/RADV b9049 | Qwen3-Coder 30B-A3B UD-Q4_K_XL | 96.76 tg128, 1320.52 pp512 | max campaign |
raw run |
n/a | max-performance r20 confirmation; previous b9010 peak was 97.24 t/s |
| Default Qwen3.6 direct path | 2026-05-07 | llama.cpp Vulkan/RADV b9049 | Qwen3.6 35B-A3B UD-Q4_K_M | 62.56 tg128, 1059.45 pp512 | benchmarks |
raw run |
chart |
clean latest-stack r20 rerun; rounds to 63 t/s |
| Fastest measured Qwen3.6 speed-first quant | 2026-05-07 | llama.cpp Vulkan/RADV b9049 | Qwen3.6 35B-A3B Q4_0 | 81.30 tg128, 1243.51 pp512 | max campaign |
raw run |
n/a | speed-first lower-quality quant; not the default all-round recommendation without a quality sanity check |
| Current normal Ollama buyer path | 2026-07-10 | Ollama 0.31.2 system service Vulkan/RADV | Qwen3.6 35B-A3B Q4_K_M | 60.57 t/s warm API generation average | benchmarks |
raw service run |
n/a | normal service-upgrade path; iGPU, vision, service restart, and full-host reboot persistence passed; later controlled local binaries did not reproduce a version-wide slowdown |
| Experimental Qwen3.6 MTP server path | 2026-05-27 | llama-server Vulkan/RADV b9360 |
Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn | 101.16 t/s best local average over six prompts; t16 repeats at 101.15 / 101.10 / 101.06 t/s; 93.29 t/s GMKtec community average on b9235 | MTP CSV |
local raw, GMKtec raw |
n/a | server/speculative result; localweights Q8-next-token-head quant; not the direct llama-bench headline |
| Experimental Gemma 4 QAT MTP server path | 2026-06-12 | llama-server Vulkan/RADV ac4cddeb0 |
Gemma 4 26B-A4B IT QAT UD-Q4_K_XL + Q4_0 MTP head | 102.69 t/s cold repeat; 107.42 t/s T3-only repeat; 110.00 t/s best repeat; 73.96 t/s no-spec baseline | MTP CSV |
cold raw, T3-only raw, warm raw |
n/a | server/speculative result with matched QAT MTP head; current Google model route; host-workload sensitive; not direct llama-bench |
| Experimental CHADROCK ROCmFP4 MTP server path | 2026-06-21 | llama-server ROCmFPX/RADV helper route |
CHADROCK ACE/SABER 35B ROCmFP4 | 140.40 and 139.93 t/s gen512 high-acceptance repeats; 127.77 t/s gen2048 check | MTP CSV |
raw helper repro |
n/a | server/speculative result with ciru-ai/ROCmFPX; prompt/acceptance-sensitive; not direct llama-bench |
| Historical May 80B Qwen-family path | 2026-05-16 | llama.cpp Vulkan/RADV b9172 | Qwen3-Next 80B-A3B UD-Q4_K_XL | 59.06 tg128, 751.70 pp512 | benchmarks |
raw r20 |
n/a | b9172 improved this 80B MoE path versus the older 54.92 t/s b8933 row |
| gpt-oss-120b loaded locally | 2026-05-07 | llama.cpp Vulkan/RADV b9049 | gpt-oss-120b MXFP4 split GGUF | 55.57 tg128, 726.99 pp512, 293.73 pp65536 r1 | max campaign |
raw run |
n/a | performance evidence only; no model-quality eval; pp65536 is one repeat |
| User-local Ollama comparator | 2026-07-02 | Ollama 0.31.1 local binary Vulkan/RADV | Qwen3.6 35B-A3B Q4_K_M | 71.82 t/s warm API generation average | benchmarks |
raw API run |
n/a | 9 warm API runs after one cold run; required OLLAMA_IGPU_ENABLE=1; faster than the earlier normal-service check, but a later controlled comparison found no version-wide 0.31.2 regression |
| Best measured Qwen3.6 server split | 2026-05-05 | Vulkan/RADV and Lemonade ROCm | Qwen3.6 35B-A3B UD-Q4_K_M | Vulkan wins 1-4 parallel; Lemonade ROCm wins 8-16 | server data |
raw sweep |
n/a | 5 reps per concurrency, 0 errors |
| HIP/Vulkan workload split | 2026-05-07 | Vulkan/RADV and ROCm/HIP from b9049 source | Qwen3.6 35B-A3B, Qwen3-Coder 30B-A3B | HIP won pp16384; Vulkan won tg128 on both local Qwen rows | max campaign |
same-source matrix |
n/a | HIP binary reports unknown build id due container git safe-directory, but source checkout was b9049 |
| Best measured Qwen3-Coder local API point | 2026-05-03 | llama-server Vulkan/RADV b9010 |
Qwen3-Coder 30B-A3B UD-Q4_K_XL | 173.16 aggregate t/s at -np 8 |
multi-user |
raw summary |
chart |
-np 16 regressed |
| 128K filled-context Qwen3.6 decode completed | 2026-05-03 | llama-server Vulkan/RADV b9010 |
Qwen3.6 35B-A3B UD-Q4_K_M | 32.23 t/s decode after 128K fill, no truncation | filled KV |
raw 128K summary |
chart |
f16 KV, synthetic long prompt |
| Real documents are slower than synthetic repeated prompts | 2026-05-03 | llama-server Vulkan/RADV b9010 |
Qwen3.6 35B-A3B and Qwen3-Next 80B-A3B | real 64K prompt ingest was 24-33% slower; decode barely changed | filled KV |
real-corpus summary |
chart |
avoids overclaiming synthetic prompt speed |
Pick the row that matches what you want to verify. The balanced coding row is the best first reproduction target for most users. The direct 100 t/s row is useful if you specifically want to verify the fastest measured 30B-class Qwen speed scout. Keep them separate: they use different model files, quants, builds, and quality tradeoffs.
AMD_VULKAN_ICD=RADV \
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json \
~/llama-cpp-upstream-2026-05-07/build-vulkan/bin/llama-bench \
-m ~/models/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf \
-fa 1 -ngl 999 -mmp 0 -p 0 -n 128 -r 20 -o csvMeasured local result: 96.76 tg128 in the max-performance b9049 campaign: raw CSV. This is the practical balanced Qwen3-Coder row. The fastest first-party Qwen3-Coder row is now 100.99 tg128 with Q4_K_S on the official b9851 Vulkan release binary, but that is a speed-first quant rather than the balanced default: raw b9851 r50. The older b9179 strict-clean 98.51 t/s row remains preserved for historical host-state context: raw b9179 r50.
Use this only if you have the exact Qwen3-30B-A3B-Instruct-2507 IQ4_XS-3.63bpw GGUF and a comparable b9467 Vulkan/RADV build. Adjust the binary and model paths to your checkout; the flags, model hash, build, and raw CSV are the reproducibility anchors. This is a direct llama-bench result, but it is not Qwen3-Coder and not the balanced default.
AMD_VULKAN_ICD=RADV \
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json \
~/llama-cpp-b9467/build-vulkan/bin/llama-bench \
-m ~/models/Qwen3-30B-A3B-Instruct-2507-IQ4_XS-3.63bpw.gguf \
-fa 1 -ngl 999 -mmp 0 -b 2048 -ub 512 -t 16 --poll 50 -p 512 -n 128 -r 50 -o csvMeasured local result: 100.04 tg128 and 1416.03 pp512 on the r50 confirmation: raw r50, model hash, scout notes.
- A first-party Beelink wall-power tokens-per-watt claim. Local amdgpu
PPTtelemetry exists, and community Corsair wall-power rows exist, but the guide still needs a Beelink wall-meter run before publishing Beelink J/token claims. - Same-machine Windows versus Linux performance. Windows LM Studio and WSL2/HIP community rows now exist, but they are not same-machine, same-model, same-shape comparisons against native Linux Vulkan/RADV.
- A default or balanced first-party 100 t/s Qwen3-Coder claim. The first-party Qwen3-Coder
Q4_K_Sspeed-first row now reaches 100.99 t/s on b9851, but the practical balancedUD-Q4_K_XLrow remains in the 96-99.6 t/s class depending on build/repeat length. A tuned community GMKtec report touched 100.0 t/s on Qwen3-Coder, and a separate first-party Qwen3-30B-A3B-Instruct-2507 IQ4_XS route reached 100.04 t/s, but those are separate evidence categories. - Production-ready NPU/FastFlowLM inference. The kernel sees
amdxdnaand/dev/accel/accel0, but XRT/FastFlowLM user-space is not installed and no local NPU LLM row is published yet. - A broadly useful DFlash/PFlash speedup on Strix Halo. The official Gemma 4 31B QAT target and matched DFlash sidecar now load and serve on b10066, but DFlash
n_max=8was 5.54% slower at 5,471 prompt tokens and 20.42% slower at 21,855 on the measured synthetic shapes because acceptance stayed low. Representative chat, coding, and reasoning prompts still need separate profiling. - A vLLM/DFlash server path that competes with
llama-serveror Ollama for a real 35B Strix Halo use case. Plain AWQ without the gated DFlash drafter was only a smoke test here at about 25 t/s. - A pinned current-HIP long-context comparison against the current Vulkan/RADV path. Historical external rocWMMA evidence exists, but upstream removed that kernel in 2026 and the older local lhl branch failed to load the current Qwen3.6 GGUFs.
- Multi-machine clustering numbers from this guide's own hardware. Community RPC and USB4 tuning data exists in
COMMUNITY_RPC.mdandUSB4_CLUSTER_TUNING.md, but it is not a local Beelink headline claim.
These numbers are local measurements from one primary machine: a Beelink GTR9 Pro with Ryzen AI MAX+ 395 and 128GB LPDDR5X-8000. Treat the headline results as directional unless your setup matches the measured run closely.
Performance depends on the exact hardware SKU, RAM configuration, BIOS UMA setting, IOMMU setting, firmware, kernel, Mesa/RADV version, ROCm version, Vulkan ICD selection, power profile, GPU clocks, thermal state, backend commit/build flags/container image, model file, quant type, model hash/path, context length, prompt length, generated token count, batch size, parallel slots, request concurrency, API endpoint, environment variables, and background system load.
If your setup differs, rerun the benchmark scripts and cite the date, command, CSV, raw log, chart, model file, and backend version with any copied claim.
| File | Purpose |
|---|---|
REPRODUCIBILITY.md |
Exact machine, BIOS/software state, commands, raw data paths, and chart generation. |
SERVER_SHOOTOUT.md |
Practical local-AI-server comparison: Ollama, llama-server, Lemonade ROCm, and vLLM candidates. |
BACKEND_CROSSOVER.md |
HIP versus Vulkan workload split: prompt processing versus token generation. |
POWER_BASELINE.md |
Local amdgpu PPT telemetry status and Beelink power-sampling caveats. |
PERFORMANCE_NOTES.md |
Narrow notes on strict-stack reruns, failed headline reproduction attempts, and useful negative model results. |
CURRENT_MODELS.md |
Current-model triage: latest model scouts, speed versus capability framing, and practical next-test value. |
UNSLOTH_STRIX_HALO.md |
Measured local fine-tuning, GGUF export, ROCm deployment, persistence, and troubleshooting path. |
data/current_test_queue.csv |
Machine-readable current test priorities, readiness, artifact size, blockers, and evidence questions. |
ROCM_VLLM_BUGWATCH.md |
Fast-moving ROCm/vLLM upstream issue and release watchlist. |
BENCHMARKS.md |
Compact benchmark source-of-truth for current README numbers. |
COMMUNITY_RESULTS.md |
Independent benchmark reports from other Strix Halo systems, kept separate from headline claims. |
COMMUNITY_FEEDBACK.md |
Community feedback loop: trust friction, public corrections, and how criticism turns into reproducible evidence. |
COMMUNITY_NIMO.md |
Nimo AI Mini PC community bundle with large-model, MTP, StepFun, Qwen 122B, Gemma 4 QAT/MTP assistant-head, and thermal context. |
COMMUNITY_RPC.md |
Community multi-node llama.cpp RPC over USB4 results, kept separate from single-machine headline claims. |
USB4_CLUSTER_TUNING.md |
Community USB4 latency tuning for active Strix Halo cluster nodes. |
THERMAL_STABILITY.md |
Scoped Corsair/Sixunited sustained-inference evidence: SCLK tradeoffs, bounded stock controls, fan-module/service checks, raw telemetry, and upstream safety work. |
ROCMFP4_CHADROCK.md |
Advanced ROCmFP4 / CHADROCK tuned-GGUF route tracking. The exact first-party reference profile averaged 141.37 t/s across three repeats at 100% acceptance, but other prompt shapes were much slower; it is not the beginner/default setup path or a direct llama-bench replacement. |
UPSTREAM_CONTRIBUTIONS.md |
Accepted upstream engineering work in llama.cpp, LocalAI, Qwen Code, OpenTelemetry GenAI, NVIDIA AICR, and vLLM GGUF tooling, with direct review links and claim boundaries. |
CONTRIBUTORS.md |
Community benchmark contributor credits and contribution path. |
CONTRIBUTING.md |
What data is most useful, which issue template to use, and how community reports become structured evidence. |
data/headline_claims.csv |
Machine-readable map from public headline claims to data, raw evidence, charts, and notes. |
data/README.md |
Structured CSV schema and raw-data conventions. |
charts/README.md |
Generated chart inventory and regeneration command. |
SHARE.md |
Copyable Reddit/HN/forum/Discord text and share links. |
SECURITY.md |
Official-source and impersonation reporting policy. |
SUPPORT.md, SERVICES.md |
How to support ongoing testing, or request scoped professional help, without changing the evidence-first benchmark policy. |
ONE_PAGE_BRIEF.md, PARTNERSHIP.md, SPONSORSHIP.md, VENDOR_DISCLOSURE.md |
Vendor/partner-facing explanation of how the technical proof layer reduces buyer adoption friction while preserving independence. |
BUYER_USE_CASES.md, SPONSOR_ROADMAP.md, TRACTION.md, BEELINK_OUTREACH.md, VENDOR_OUTREACH_PLAN.md, OUTREACH_TEMPLATES.md |
Buyer-use-case, roadmap, public-evidence, and outreach support docs. |
- 20-Second Summary
- Use This Guide If
- Quick Start (6 Steps)
- Qwen3.8 On Strix Halo
- Setup Script
- What You Can Run: Quick Snapshot
- Use This If You Want
- Community-Tested Rules Of Thumb
- Results Wanted
- Community Results
- Best Current Setup Tested Here
- Headline Evidence
- Reproduce One Headline Result
- Not Yet Proven Here
- Do Not Copy These Claims Without Matching Setup
- Documentation Map
- Hardware
- What You Can Run
- Benchmark Results
- Backend Decision Guide
- Server Shootout
- Phase 1: BIOS Configuration
- Phase 2: Ubuntu 24.04 Installation
- Phase 3: Kernel Configuration
- Phase 4: Performance Tuning
- Phase 5: Ollama Setup (Vulkan)
- Phase 6: Benchmarking
- Phase 7: ROCm with llama.cpp (Containers)
- Phase 8: vLLM Serving
- Phase 9: Multi-Node Clustering (RDMA)
- Phase 10: SSH and Remote Access
- Vulkan Driver Comparison
- Key Findings and Corrections
- Known Issues
- Troubleshooting
- Kernel and ROCm Compatibility
- Power Measurement Status
- Testing Checklist
- Model Recommendation Guide
- Cost: Local vs Cloud
- Buying Guide
- Glossary
- FAQ
- Community Resources
- Credits and References
- Contributing
- Changelog
- License
The count is 10 described owner systems plus 3 independently attributable
external evidence sources. One physical owner system counts once regardless
of how many results it contributes; separate physical machines count
separately even when they are the same product or belong to the same
contributor. External packages whose unique hardware identity cannot be safely
proved remain labelled as sources. This makes the public 13 systems/sources
claim auditable without treating every row as an equal-quality benchmark.
For a reviewer-friendly cross-OEM view of what each source proves, how the evidence classes differ, and which validation is still missing, see the AMD Strix Halo System Evidence Matrix. A machine-readable version is available at data/system_evidence_matrix.csv.
| Evidence source | Systems/sources counted | Running total | Evidence |
|---|---|---|---|
| Primary Beelink GTR9 Pro | 1 | 1 | First-party raw evidence throughout this repository |
| Corsair AI Workstation 300 fleet | 3 | 4 | COMMUNITY_RESULTS.md, THERMAL_STABILITY.md, data/community_results.csv |
| GMKtec EVO-X2 native contributor system | 1 | 5 | COMMUNITY_RESULTS.md, data/raw/2026-05-14/community-gmktec-native-issue16/ |
| GMKtec EVO-X2 tuned Reddit report | 1 | 6 | COMMUNITY_RESULTS.md, raw note |
| Minisforum MS-S1-Max | 1 | 7 | COMMUNITY_RESULTS.md, raw report |
| Nimo AI Mini PC | 1 | 8 | COMMUNITY_NIMO.md |
| Second Beelink GTR9 Pro owner stack | 1 | 9 | COMMUNITY_RESULTS.md, raw report |
| ciru-ai GMKtec EVO-X2 NixOS/NPU artifact | 1 | 10 | COMMUNITY_RESULTS.md, source artifact |
| Minix Elite ER939 Ai | 1 | 11 | COMMUNITY_RESULTS.md, raw report |
| Third independent Beelink GTR9 Pro unit | 1 | 12 | COMMUNITY_RESULTS.md, raw report |
| Kyanite Labs GMKtec EVO-X2 Qwen3.8 package | 1 | 13 | COMMUNITY_RESULTS.md, scope note, source artifact |
This table is a product/context map, not an endorsement list or the source of the thirteen-source count above. Rows without linked community evidence are useful hardware context only.
| System | CPU | GPU | RAM | Notes |
|---|---|---|---|---|
| Beelink GTR9 Pro | Ryzen AI MAX+ 395 | Radeon 8060S (40 CU) | 128GB LPDDR5X-8000 | This guide's primary test system |
| Corsair AI Workstation 300 | Ryzen AI MAX+ 395 | Radeon 8060S (40 CU) | 128GB LPDDR5X-8000 | Three community systems reproduced the Qwen3-Coder path |
| Framework Desktop | Ryzen AI MAX+ 395 | Radeon 8060S (40 CU) | 128GB LPDDR5X-8000 | Used by kyuz0, lhl |
| GMKtec EVO-X2 | Ryzen AI MAX+ 395 | Radeon 8060S (40 CU) | 96GB or 128GB LPDDR5X-8000 | Native 96GB community rows reproduce Qwen3.6 and add a stock Gemma 4 direct control; a separate corrected 96GB Kyanite package reaches 262K-class Qwen3.8 retrieval on a patched HIP route; a tuned Reddit report touched 100.0 t/s on Qwen3-Coder Q4_K_S; pablo-ross guide |
| Minisforum MS-S1-Max | Ryzen AI MAX+ 395 | Radeon 8060S (40 CU) | 128GB LPDDR5X | Windows LM Studio community serving report imported; not a same-shape Linux comparison |
| Nimo AI Mini PC | Ryzen AI MAX+ 395 | Radeon 8060S (40 CU) | 128GB LPDDR5X | Community issue #4 bundle imported; adds compact-chassis large-model serving, MTP, StepFun/Qwen 122B, Gemma 4 QAT/MTP assistant-head follow-up data, and thermal context |
| HP ZBook Ultra G1a | Ryzen AI MAX+ 395 | Radeon 8060S (40 CU) | 128GB LPDDR5X-8000 | Workstation laptop |
| Component | Spec |
|---|---|
| CPU | AMD Ryzen AI MAX+ 395 (16 cores / 32 threads, Zen 5) |
| GPU | Radeon 8060S (gfx1151, RDNA 3.5, 40 CUs) |
| RAM | 96GB or 128GB unified LPDDR5X-8000 depending on vendor; primary measured system is 128GB (~215 GB/s measured, 256 GB/s theoretical) |
| NPU | RyzenAI-npu5 (XDNA 2) |
Why this hardware? 96GB/128GB unified memory shared between CPU and GPU means you can run 70B+ models entirely on the GPU -- something an RTX 4090 (24GB VRAM) cannot do. You trade raw bandwidth (~215 GB/s vs ~1 TB/s on this Beelink) for the ability to run larger model artifacts (not automatically higher task quality) on one compact machine. Price changes quickly by vendor; check the Buying Guide before making a purchase decision.
Real-world generation speeds measured on the Beelink GTR9 Pro, primarily with Vulkan/RADV. Speeds marked with * are via llama-bench direct; others are via Ollama unless noted. Use this table to choose a first model, then follow the evidence links above before copying a benchmark claim.
| Model | Size | Type | Generation Speed | Use Case |
|---|---|---|---|---|
| Qwen3-0.6B (Q8_0) | 0.8 GB | Dense | 266 t/s * | Ultra-fast tiny model |
| Llama 2 7B | 3.8 GB | Dense | 48-52 t/s | Testing, lightweight tasks |
| Qwen2.5-VL 7B | 6.0 GB | Vision | 21.4 t/s | Image understanding |
| LFM2.5-VL 1.6B (Q4_0) | 1.19 GB with projector | Vision | functional local pass | Lightweight official-GGUF image route; one-image smoke, not a speed/quality headline |
| Qwen3-ASR 0.6B (Q8_0) | 0.95 GB with projector | Speech | functional local pass | Short offline English transcription; experimental audio path |
| LFM2.5 8B-A1B (Q4_K_M) | 5.1 GB | MoE | 170.0 t/s * | Fastest current small-MoE scout; not a 30B-class replacement |
| Gemma 4 26B-A4B IT QAT (UD-Q4_K_XL) | 14.2 GB | MoE | 74.8 t/s direct; 102.7-110.0 t/s MTP server | Current practical Google-model route; direct row is llama-bench, MTP row is server/speculative |
| Qwen3-30B-A3B-Instruct-2507 (IQ4_XS) | 13.9 GB | MoE | 100.0 t/s * | Fastest direct 30B-class Qwen row; general-instruct route, not Qwen3-Coder |
| Qwen3-Coder 30B-A3B (Q4_K_S) | 17.5 GB | MoE | 101.0 t/s * | Fastest measured coding speed; speed-first quant, not the balanced default |
| Qwen3-Coder 30B-A3B (UD-Q4_K_XL) | 17.7 GB | MoE | 97 t/s * | Measured balanced-quant candidate; no matched quality score; current b9049 measured 96.76 t/s and previous b9010 peak was 97.24 t/s |
| Qwen3.6 35B-A3B (Q4_0) | 19.7 GB | MoE | 81 t/s * | Fastest measured Qwen3.6 speed-first quant; use a balanced quant if quality matters more than raw speed |
| Qwen3.6 35B-A3B (Q4_K_M / UD-Q4_K_M) | 20-22 GB | MoE | 63 t/s * | Balanced direct path; separate speed-first/alternate quants reach higher but need quality sanity |
| Qwen3.5 35B-A3B | 23 GB | MoE | 48-65 t/s | General purpose, coding (65 with measured direct llama.cpp builds) |
| Qwen3-Coder 30B-A3B (Q8_0) | 32 GB | MoE | 51 t/s | Coding (higher-bit quant; no matched quality evaluation) |
| Qwen3-Coder-Next | 51 GB | MoE (80B total / 3B active) | 38-39 t/s | Coding MoE; size is the measured artifact, not parameter count |
| Llama 3.1 70B (Q4_K_M) | 42 GB | Dense | 4.7-4.9 t/s | Dense 70B capacity; artifact exceeds 24GB VRAM without offload |
| Llama 4 Scout 109B (Q4_K_M) | 61 GB | MoE | 18.3 t/s * | 109B artifact on a mini PC; exceeds 24GB VRAM without offload |
| Nemotron 3 Nano 30B-A3B (IQ4_XS) | 18.2 GB | MoE | 76.0 t/s * | Practical NVIDIA Nemotron 30B-class route |
| DeepSeek V4 Flash 284B (UD-IQ2_XXS) | 90.9 GB | MoE | 13.3 t/s * | Largest current direct ordinary-GGUF capacity proof; low-bit quant, not a quality recommendation |
| Nemotron 3 Super 120B-A12B (UD-IQ4_XS) | 64.5 GB | MoE | 18.4 t/s * | Current 120B-class GGUF route on one 128GB Strix Halo |
| gpt-oss-120b MXFP4 | 63.4 GB | MoE | 55.6 t/s * | 117B-parameter open-weight model; local load and long-context speed check |
| Qwen3-Next 80B-A3B (UD-Q4_K_XL) | 42.90 GiB | MoE | 59 t/s * | 80B model, 256K context -- a separate 80B MoE route |
| Kimi K2.5 1T (4-node cluster) | 375 GB artifact; 512GB aggregate RAM | MoE | distributed | AMD technical article |
Benchmarks below were run on 2026-03-20, 2026-03-21, 2026-04-26, 2026-05-03, 2026-05-07, 2026-05-16, 2026-05-26, 2026-05-27, 2026-05-31, 2026-06-01, 2026-06-02, 2026-06-05, 2026-06-07, 2026-06-11, 2026-06-12, 2026-06-21, 2026-06-30, and 2026-07-02. Primary benchmark system: Beelink GTR9 Pro. Recorded local runs used kernel 6.19.4, Mesa RADV 26.0.2-26.1.3 where captured, AMDVLK removed, and tuned accelerator-performance where captured; individual raw directories and CSV rows are the source of truth for exact run metadata, and some older or scout rows intentionally record missing metadata as not recorded. Before new benchmarks, record and verify the selected power/background policy; require tuned only for a tuned-profile reproduction.
These rows are included because they answer practical setup questions: which model to try first, which backend removes the most friction, which paths are only experimental, and which results are strong enough to cite.
These generated SVGs summarize the current structured benchmark data. The CSV files in data/ and raw logs in data/raw/ remain the source of truth; regenerate charts with python3 scripts/generate_charts.py.
| Multi-user serving | Long-context prompt scaling |
|---|---|
 |
 |
| Filled-KV decode | KV-cache quantization tradeoff |
|---|---|
 |
 |
| Real versus synthetic prompts | Backend spot check |
|---|---|
 |
 |
Qwen3.6-35B-A3B (Q4_K_M, ~20-23GB, MoE):
| Prompt Tokens | Prompt Eval | Generation | Notes |
|---|---|---|---|
| 25 | 944 t/s | 71.8 t/s | 2026-07-02 user-local Ollama 0.31.1 binary sanity check; required OLLAMA_IGPU_ENABLE=1; 9-run warm average after one cold run |
| 19 | 158 t/s | 50.5 t/s | Controlled 2026-05-07 API warm average across 10 runs; matches 0.21.2 |
| 20 | 163 t/s | 45.6 t/s | Older result, superseded by controlled API run |
| 22 | 174 t/s | 45.4 t/s | Older result, superseded by controlled API run |
Qwen3.5-35B-A3B (Q4_K_M, ~23GB, MoE -- Ollama 0.20.4):
| Prompt Tokens | Prompt Eval | Generation | vs Previous (Mesa 26.0.1) |
|---|---|---|---|
| 14 | 121.3 t/s | 48.0 t/s | tg +4.8% |
| 23 | 182.3 t/s | 47.5 t/s | tg +4.4% |
| 122 | 456.7 t/s | 47.4 t/s | tg +4.2% |
Qwen3-Coder 30B-A3B (Q8_0, ~32GB, MoE):
| Prompt Tokens | Prompt Eval | Generation | Notes |
|---|---|---|---|
| 12 | 118.3 t/s | 51.4 t/s | Fastest via Ollama |
| 21 | 205.2 t/s | 51.3 t/s | Higher-bit quant than Q4_K_M; quality not evaluated here |
Qwen3-Coder-Next (~51GB artifact, 80B total / 3B active MoE):
| Prompt Tokens | Prompt Eval | Generation | vs Previous |
|---|---|---|---|
| 12 | 90.7 t/s | 39.1 t/s | tg +2.9% |
| 21 | 129.5 t/s | 38.4 t/s | tg +3.8% |
| 120 | 301.2 t/s | 37.9 t/s | NEW |
Other Models:
| Model | Size | Prompt Tokens | pp (t/s) | tg (t/s) |
|---|---|---|---|---|
| Llama 2 7B | 3.8 GB | 24 | 384.6 | 52.0 |
| Qwen2.5-VL 7B | 6.0 GB | 23 | 81.7 | 21.4 |
| Qwen3.5 35B (no-think) | 23 GB | 14 | 127.1 | 47.4 |
Llama 3.1 70B (Q4_K_M, 42GB, Dense -- the "doesn't fit on RTX 4090" showcase):
| Prompt Tokens | Prompt Eval | Generation | Notes |
|---|---|---|---|
| 14 | 22.1 t/s | 4.9 t/s | Cold start |
| 23 | 36.8 t/s | 4.8 t/s | Realistic chat |
| 122 | 79.6 t/s | 4.7 t/s | Long prompt |
A rough weight-streaming estimate is 215 GB/s divided by 42 GB, or about 5.1 t/s, compared with the observed 4.8 t/s. This simplified model omits compute, KV/cache traffic and implementation effects; it is not a proven ceiling or evidence that optimization cannot help. The artifact cannot fit wholly in an RTX 4090's 24GB VRAM; partial CPU offload is a separate memory/latency tradeoff.
The historical combined Mesa/power-profile change was reported alongside higher generation rates. It does not isolate a tuned-only effect or establish a universal percentage across models.
Single-user llama-bench tells you the ceiling for one stream. For a real local API box, the more practical question is what happens when multiple tools or users hit llama-server at the same time.
Qwen3.6-35B-A3B (UD-Q4_K_M, Vulkan RADV, llama.cpp b9010, continuous batching, 4096 context tokens per slot):
-np |
Concurrent Requests | Aggregate Generation | Avg per Request | Mean TTFT | Mean request decode interval | Notes |
|---|---|---|---|---|---|---|
| 1 | 1 | 59.21 t/s | 59.21 t/s | 0.117 s | 16.1 ms | Server/API baseline |
| 2 | 2 | 92.21 t/s | 46.11 t/s | 0.198 s | 20.3 ms | Good scaling |
| 4 | 4 | 130.81 t/s | 32.71 t/s | 0.237 s | 29.0 ms | Strong batching gain |
| 8 | 8 | 161.98 t/s | 20.25 t/s | 0.307 s | 47.4 ms | Practical sweet spot |
| 16 | 16 | 165.98 t/s | 10.38 t/s | 0.547 s | 92.9 ms | Throughput plateau |
Takeaway: continuous batching makes Strix Halo look much stronger as a local API server than single-user tg numbers suggest.
-np 8delivers about 2.7x the-np 1aggregate throughput while keeping TTFT around 0.3 seconds.-np 16works with no errors in this test, but aggregate throughput barely improves while per-user speed drops sharply.
Qwen3-Coder 30B-A3B (UD-Q4_K_XL, Vulkan RADV, llama.cpp b9010, continuous batching, 4096 context tokens per slot):
-np |
Concurrent Requests | Aggregate Generation | Avg per Request | Mean TTFT | Mean request decode interval | Notes |
|---|---|---|---|---|---|---|
| 1 | 1 | 90.20 t/s | 90.20 t/s | 0.079 s | 10.6 ms | Server/API baseline |
| 2 | 2 | 121.65 t/s | 60.83 t/s | 0.133 s | 15.5 ms | Good scaling |
| 4 | 4 | 157.41 t/s | 39.36 t/s | 0.207 s | 24.0 ms | Strong batching gain |
| 8 | 8 | 173.16 t/s | 21.65 t/s | 0.382 s | 43.5 ms | Practical sweet spot |
| 16 | 16 | 129.56 t/s | 8.10 t/s | 0.571 s | 119.9 ms | Regression |
Coding server takeaway:
-np 8is also the best measured setting for Qwen3-Coder, but here-np 16is actively worse. More parallel slots are not automatically better.
Raw data: data/multi_user.csv, data/raw/2026-05-03/multi-user/, and data/raw/2026-05-03/multi-user-coder/.
Historical Qwen3.5-35B-A3B comparison (2026-03-21): b8298 to b8460 improved pp512 by about 24% and tg128 by about 25% on the recorded Vulkan pair. The build includes changes such as PR #19625, PR #20551 and PR #20334, but this endpoint comparison does not isolate their individual contributions.
Important caveats:
- The 25% observation is scoped to that Qwen3.5 pair. Qwen3-Coder's recorded 86.81→87.11 t/s pair did not show that gain. Small changes on dense models do not prove a bandwidth ceiling.
- For kyuz0's containers, inspect the actual binary revision and image digest. A tag or rebuild policy does not prove an already-installed container contains a particular update.
- WARNING: AMDVLK silently overrides RADV. If AMDVLK is installed, its
/etc/vulkan/icd.d/amd_icd64.jsontakes priority over RADV. This reduced pp speed by about 39% in the recorded pair (1080 -> 660 pp512) without any visible error. Always setAMD_VULKAN_ICD=RADVor uninstall AMDVLK entirely:sudo dpkg -r amdvlk && sudo rm -f /etc/vulkan/icd.d/amd_icd64.json. Check your driver: RADV shows(RADV STRIX_HALO) (radv)withshared memory: 65536in llama-bench output. AMDVLK shows(AMD open-source driver)withshared memory: 32768. We originally reported this as a llama.cpp regression -- it wasn't.
Qwen3.5-35B-A3B (Q4_K_M, 19.9GB, MoE) -- the biggest improvement:
| Build | Driver | pp128 | pp512 | tg128 | vs old RADV |
|---|---|---|---|---|---|
| b8460 (newer March build) | RADV | 623 | 1080 | 64.85 | pp +24%, tg +25% |
| b8460 (newer March build) | AMDVLK | 521 | 663 | 64.10 | pp -24%, tg +23% |
| b8298 (kyuz0) | RADV | 583 | 868 | 52.06 | baseline |
| b8298 (kyuz0) | AMDVLK | 479 | 576 | 56.08 |
Beginner rule: use RADV for Vulkan. Do not install AMDVLK. In the newer b8460 Vulkan comparison, RADV is faster than AMDVLK on both pp (+63%) and tg (+1.2%), and AMDVLK can silently hijack your Vulkan driver. Advanced note: ROCm/HIP is a different backend, not a Vulkan driver. HIP can still be worth testing for long prompts, RAG ingest, and other prompt-processing-heavy workloads.
Extended context scaling (b8460 RADV):
| pp512 | pp2048 | pp4096 | pp8192 | Drop at 8K |
|---|---|---|---|---|
| 1080 | 1057 | 1049 | 1049 | -3% |
pp is virtually flat from 512 to 8192 tokens. Only 3% drop at 8K context.
Qwen3-Coder 30B-A3B (Q4_K_S speed-first and UD-Q4_K_XL balanced, MoE):
| Build | Driver | pp512 | tg128 | Notes |
|---|---|---|---|---|
| b9851 | RADV | 1423 | 100.99 | Official release binary; Q4_K_S speed-first quant; not the balanced default |
| b9179 | RADV | 1396 | 98.51 | Q4_K_S speed-first quant, strict-clean r50 confirmation |
| b9049 | RADV | 1321 | 96.76 | 2026-05-07 max-performance guide-flags r20 confirmation |
| b9010 | RADV | 1346 | 97.24 | Controlled 2026-05-03 two-run r20 average |
| b8460 | RADV | 1342 | 87.11 | Previous headline |
| b8298 (kyuz0) | RADV | 1350 | 86.81 | ~same (model was already at ceiling) |
A controlled May 2026 rerun moved the balanced Qwen3-Coder 30B headline from 87 t/s to 97 t/s on b9010 Vulkan RADV. The 2026-05-07 b9049 max-performance campaign measured 96.76 t/s on UD-Q4_K_XL. A later strict-clean b9179 run measured 98.51 t/s with Q4_K_S, and the official b9851 Vulkan release binary lifted the same speed-first quant to 100.99 t/s. This is still not the default balanced row.
Gemma 4 26B-A4B historical non-QAT baseline (UD-Q4_K_M, 15.7GB, MoE) -- tested on b8933 (earliest build with Gemma 4 support):
| Build | Driver | pp512 | tg128 | Notes |
|---|---|---|---|---|
| b8933 | RADV | 1142 | 48.46 | early non-QAT Gemma 4 MoE baseline |
This row is historical and non-QAT. The current practical Gemma 4 route in this guide is Gemma 4 26B-A4B IT QAT UD-Q4_K_XL, which measured 74.80 t/s direct and 102.69 cold / 107.42 T3-only / 110.00 best-repeat t/s through a matched MTP
llama-serverroute. Keep the direct row and server/speculative row separate.The older non-QAT Gemma row was slower than the recorded Qwen MoE routes. Attention dimensions, mixed attention and active-parameter traffic are possible contributors, alongside runtime implementation. These unmatched rows do not isolate a cause or rule out llama.cpp improvements; the later non-QAT b9851 result reached 55.45 t/s.
KV precision caveat: an external KL experiment reported q8_0 KV KL 0.377 for Gemma 26B-A4B versus 0.108 for Gemma 31B, about 3.49×. It compared output distributions against an f16-KV reference; this is not a factor of downstream task-quality loss and not a comparison with Qwen. Keep f16 KV as this guide's conservative tested profile; lower-precision KV needs artifact/workload-specific qualification.
Llama 4 Scout 109B (Q4_K_M, 60.9GB, MoE -- 109B total params, 17B active):
| Build | Driver | pp512 | tg128 | Notes |
|---|---|---|---|---|
| b8933 | RADV | 331 | 18.32 | 109B model running on a mini PC |
The recorded 109B artifact ran at 18.3 t/s on the 128GB system and cannot fit wholly in 24GB VRAM. A nominal 17B-active-at-4-bit bandwidth estimate gives roughly 25 t/s at 215 GB/s, but omits mixed-quant bytes, KV/cache traffic and compute; it is a rough model, not an established performance ceiling.
Qwen3-Next 80B-A3B (UD-Q4_K_XL, 42.90 GiB, MoE -- 80B total params, 3B active, 256K context):
| Build | Driver | pp512 | tg128 | Notes |
|---|---|---|---|---|
| b9172 | RADV | 752 | 59.06 | Latest-stack r20 confirmation; historical May 80B Qwen-family path |
| b8933 | RADV | 657 | 54.92 | 80B model at 55 t/s |
80 billion parameters running at 59 t/s on a mini PC. This measured Qwen3-family MoE has 80B total with only 3B active parameters and a 256K context window. Despite being 42.90 GiB of reported llama-bench model bytes (46.06 decimal GB), the MoE routing keeps only 3B params active per token. The measured route reached 59 t/s versus 38 t/s for the separate Qwen3-Coder-Next 80B-A3B artifact/runtime row. This is not a matched architecture or quality comparison. The 2026-05-16 b9172 check improved this row, while Qwen3-Coder, Qwen3.6, and gpt-oss did not improve on the same latest-stack rerun.
Qwen3.6-35B-A3B (Q4_K_M, 19.9GB, MoE -- drop-in upgrade from Qwen3.5, released April 2026):
| Build | Driver | pp512 | tg128 | Notes |
|---|---|---|---|---|
| b9049 | RADV | 1059 | 62.56 | Clean 2026-05-07 latest-stack r20 rerun |
| b8460 | RADV | 1064 | 63.76 | Same speed as Qwen3.5 |
| b9010 | RADV | 1109 | 63.06 | UD-Q4_K_M controlled rerun; plain Q4 blob not loadable by upstream b9010 |
| b8933 | RADV | 1040 | 63.66 | No regression between builds |
Qwen3.6 is a measured runtime-compatible successor on these routes; this guide has not established a general coding/reasoning-quality improvement over Qwen3.5. Older April data showed a 13% UD-Q4_K_M penalty, but the controlled May b9010 and b9049 reruns did not reproduce that large gap. Prefer plain Q4_K_M when you have a direct-compatible GGUF, but treat the old "UD is always 13% slower" warning as superseded until same-build plain-vs-UD is rerun.
The March/May b8460 route on kernel 6.19.4 required
HSA_OVERRIDE_GFX_VERSION=11.5.1 + HSA_ENABLE_SDMA=0. Those variables
belong to this dated comparison and are not the current default. Current
ROCm builds that detect the iGPU natively as gfx1151 should be run without
a global HSA architecture override.
| Build | pp128 | pp512 | tg128 | Notes |
|---|---|---|---|---|
| b8460 (newer March build, kernel 6.19.4) | 547 | 1047 | 54.67 | tg +14% vs b8301 |
| b8301 (self-compiled, kernel 6.19.4) | 542 | 1059 | 47.87 | old build |
| b8301 (self-compiled, kernel 6.18.14) | 488 | 996 | 48.80 | previous best |
ROCm's historical pair improved from 47.87 to 54.67 tg128 (+14%). Vulkan RADV remained faster in the recorded short-context pair: 1080 versus 1047 pp512 (+3%) and 64.85 versus 54.67 tg128 (+19%). These endpoint deltas cannot be decomposed into generic and Vulkan-specific optimization percentages without isolated A/B evidence. HIP batching and prompt-heavy workloads remain separate tests; the removed rocWMMA kernel is historical evidence.
ROCm HIP spot check (2026-05-03, b8460 HIP build):
| Model | Quant | ROCm pp512 | ROCm tg128 | Vulkan Reference |
|---|---|---|---|---|
| Qwen3.6 35B-A3B | UD-Q4_K_M | 1186 | 52.7 | Vulkan b9010: 1109 pp, 63.1 tg |
| Qwen3-Coder 30B-A3B | UD-Q4_K_XL | 1285 | 73.7 | Vulkan b9010: 1346 pp, 97.2 tg |
This HIP run required
LD_LIBRARY_PATH=/usr/local/lib/ollama/rocmplus the HSA override and emitted a missingTensileLibrary_lazy_gfx1151.datwarning. Treat it as a ROCm HIP baseline, not a tuned rocBLASLt/rocWMMA result. Vulkan RADV remains the recommended short-context generation backend.
Build version matters enormously:
| What we tested | pp512 | tg128 | Lesson |
|---|---|---|---|
| Ollama Vulkan RADV (b8298) | ~457 (via API) | 47.4 | Ollama adds overhead |
| llama-bench RADV (b8298) | 868 | 52.06 | Eliminating Ollama helps |
| llama-bench RADV (b8460) | 1080 | 64.85 | Updating llama.cpp = +25% |
| ROCm HIP (b8301, HSA fix) | 1059 | 47.87 | Old build, unfair comparison |
| ROCm HIP (b8460, HSA fix) | 1047 | 54.67 | ROCm got +14% tg from same update |
The early Qwen3.5 pair showed a substantial build-to-build gain. That observation supports checking the exact runtime before tuning, but does not promise that newer builds improve every MoE or that dense models cannot improve.
Batch size and ubatch tuning results (b8298, for reference):
We swept batch sizes 64-2048 and ubatch sizes 32-1024. Result: default 512 is optimal. No headroom via tuning -- the improvement came from updating the build.
Pinned candidate build: llama.cpp v0.4.1 with Vulkan
This is a source/CLI-aligned candidate example, not a locally qualified replacement
for the measured builds. Use a new directory; do not overwrite a working checkout.
The v0.4.1 parsers use --load-mode instead of the removed load flags
(upstream change). auto below
is a starting policy, not a measured optimum. Historical commands retain their
original flags and build IDs.
git clone --branch v0.4.1 --depth 1 https://github.com/ggml-org/llama.cpp llama.cpp-v0.4.1-vulkan
cd llama.cpp-v0.4.1-vulkan
CC=/usr/bin/gcc CXX=/usr/bin/g++ cmake -B build -S . \
-DGGML_VULKAN=ON \
-DCMAKE_BUILD_TYPE=Release \
-G "Unix Makefiles"
cmake --build build -j$(nproc)
# Inspect this binary's CLI before the first candidate smoke
./build/bin/llama-bench --help
# Candidate benchmark (not a published performance result)
AMD_VULKAN_ICD=RADV ./build/bin/llama-bench \
-m ~/models/your-model.gguf \
-fa on -ngl 999 --load-mode auto -p 512 -n 128# Reproduce only the dated b8460 / kernel 6.19.x evidence:
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export HSA_ENABLE_SDMA=0
export ROCBLAS_USE_HIPBLASLT=1Do not add this override globally by default for current ROCm images. First run
llama-cli --list-devices; if the current stack already reports gfx1151,
leave HSA_OVERRIDE_GFX_VERSION unset.
Llama 2 7B (Q4_K_M, 3.8GB, Dense):
| Driver | pp128 | pp512 | pp1024 | tg128 |
|---|---|---|---|---|
| RADV | 1154 | 1377 | 1356 | 48.12 |
| AMDVLK | 335 | 327 | 325 | 48.02 |
AMDVLK was 3-4X slower on pp for the recorded Llama 2 7B pair. A 2 GiB allocation limit is a possible contributor, not an isolated causal measurement; the measured beginner route uses RADV.
Qwen3-0.6B (Q8_0, 762MB, Dense) -- maximum throughput:
| Driver | pp128 | pp512 | tg128 |
|---|---|---|---|
| RADV | 10,313 | 13,112 | 266 |
Historical note (March 2026): The measured b8460/kernel 6.19.4 combination detected the iGPU incorrectly without
HSA_OVERRIDE_GFX_VERSION=11.5.1andHSA_ENABLE_SDMA=0. Preserve those variables when reproducing that exact row. Current ROCm images with nativegfx1151support should not inherit the workaround.
Previous results on kernel 6.18.14 (for reference -- these worked):
| Build | Model | pp128 | pp512 | tg128 |
|---|---|---|---|---|
| Self-compiled b8301, FA on, -mmp 0 | Qwen3.5-35B-A3B Q4_K_M | 488 | 996 | 48.8 |
| kyuz0 b8298, FA on | Qwen3.5-35B-A3B Q4_K_M | 306 | 520 | 55.3 |
| kyuz0 b8298, FA off | Qwen3.5-35B-A3B Q4_K_M | 352 | 524 | 53.8 |
| kyuz0 b8189, FA + hipBLASLt | Llama 2 7B Q4_K_M | 1163 | 1261 | 45.07 |
Vulkan llama-bench Direct (kyuz0 containers, b8298) -- March 2026:
| Driver | Model | pp128 | pp256 | pp512 | pp1024 | tg128 |
|---|---|---|---|---|---|---|
| RADV | Qwen3.5-35B-A3B Q4_K_M | 503.67 | - | 858.88 | - | 52.15 |
| AMDVLK | Qwen3.5-35B-A3B Q4_K_M | 477.28 | - | 575.59 | - | 55.54 |
| RADV | Llama 2 7B Q4_K_M | 1153.53 | 1364.45 | 1377.18 | 1355.88 | 48.12 |
| AMDVLK | Llama 2 7B Q4_K_M | 334.50 | 337.96 | 327.35 | 325.33 | 48.02 |
Critical finding (b8298): The recorded AMDVLK Llama 2 7B pp result was 3-4X slower. Its reported 2 GiB single-buffer limit is a hypothesis for investigation, not a proven sole cause. On MoE models, AMDVLK was slightly faster on tg (+6.5%) with b8298, but this advantage disappeared with b8460 -- see the key build comparison. For beginners: keep AMDVLK removed and use RADV for Vulkan.
Vulkan RADV vs ROCm HIP (same build b8460, Qwen3.5-35B-A3B):
| Metric | Ollama (b8298) | Vulkan RADV (b8460) | ROCm HIP (b8460) | Best |
|---|---|---|---|---|
| pp512 | ~457 | 1080 | 1047 | Vulkan RADV |
| tg128 | 47.4 | 64.85 | 54.67 | Vulkan RADV |
Vulkan RADV wins on both pp512 and tg128 for this Qwen3.5 b8460 short-context pair. ROCm works on kernel 6.19.x with the HSA override fix, and newer spot checks show HIP can win longer prompt-processing rows. Use
llama-benchorllama-serverdirectly instead of Ollama to avoid the measured short-context overhead on Qwen3.6. As upstream context rather than a local measurement, openollama/ollama#15601, rechecked 2026-08-30, reports about 34 t/s through Ollama's b7437-era vendoredllama.cppversus 52-56 t/s standalone (roughly a 56% gap) because the vendored tree lacks newer Vulkan Wave32 FlashAttention and graphics-queue changes; whether Ollama 0.33.2 closed that gap is unknown.
Based on our measurements and lhl's detailed testing:
| Backend | Best For | pp (relative) | tg (relative) | Context Scaling | Setup Difficulty |
|---|---|---|---|---|---|
| Ollama + Vulkan RADV | General use, chat | Good | Good | Degrades at 8K+ | Easiest |
| llama.cpp + Vulkan RADV (container) | Best-tested generation-heavy GGUF path | Best-tested in measured short-context rows | Best-tested short-context generation | Degrades at 8K+ | Easy |
| llama.cpp + Vulkan AMDVLK | Not recommended | Slower than RADV on b8460 | Slower on the recorded dense pair; cause not isolated | Degrades at 8K+ | Easy |
| ROCm HIP | Batch processing | Excellent | Good | Poor at 32K+ | Medium (needs HSA fix on 6.19.x) |
| vLLM (TheRock) | API serving | Good | Good | Good | Hard |
rocWMMA route removed (2026-08-30 note): the earlier "ROCm + rocWMMA (tuned) for long context" row is retracted. This guide measured a 73% regression on the ROCm 7.2 rocWMMA container (see Known Issues), and upstream
llama.cppremoved the rocWMMA FlashAttention kernel entirely in PR #26046 (merged 2026-07-24); HIP builds now always use the newer MMA kernel. External rocWMMA columns in older tables below are historical evidence only.
Capacity and bandwidth are hardware context, not matched inference benchmarks.
| Hardware | Memory capacity context | Workload evidence in this guide |
|---|---|---|
| RTX 4090 / RTX 3090 | 24GB dedicated VRAM; host offload is a separate route | Older 100-122 / 100-112 t/s ranges lacked exact matched artifact/build/workload provenance and are not a buying-performance comparison |
| Apple Mac Studio | Configuration-dependent unified memory; verify exact SKU | No matched first-party comparison here |
| Beelink GTR9 Pro tested system | 128GB physical unified RAM; OS/KV/runtime need headroom | Direct Qwen3-Coder Q4_K_S 100.99 tg128 and gpt-oss-120b 55.57 tg128 are separate model/quant/build rows |
| NVIDIA DGX Spark | 128GB unified memory | External 52-56 t/s gpt-oss-120b reports are contextual; matching a model family does not establish identical artifact/backend/prompt/cache conditions |
Do not rank these systems from unmatched ranges or capacity alone. The earlier Beelink $4,349 US snapshot was checked August 29, 2026; other older price mentions are not a current exact-SKU comparison. Use the dated buyer snapshots and verify complete-system configuration, region and availability before a purchase decision. External context: NVIDIA, Framework discussion, lhl's tests.
Local prompt-processing scaling on the May 2026 b9010 Vulkan RADV stack:
| Model | 4K pp | 8K pp | 16K pp | 32K pp | 64K pp | Notes |
|---|---|---|---|---|---|---|
| Qwen3.6 35B-A3B UD-Q4_K_M | 1082 | 1089 | 1025 | 909 | 740 | 68% of 4K speed retained at 64K |
| Qwen3-Next 80B-A3B UD-Q4_K_XL | 742 | 736 | 700 | 645 | 544 | 73% of 4K speed retained at 64K |
Local result: long-prompt ingestion remains usable through 64K on both tested MoE models. This table measures prompt processing, not generation after a fully occupied KV cache. Raw data:
data/long_context.csvanddata/raw/2026-05-03/long-context/.
Filled-KV decode through llama-server on the same stack:
| Model | Prompt | KV | Prompt Eval | Decode After Fill | Wall Time |
|---|---|---|---|---|---|
| Qwen3.6 35B-A3B | 32K | f16 | 1217 t/s | 51.0 t/s | 29.5 s |
| Qwen3.6 35B-A3B | 32K | q4_0 | 1049 t/s | 56.0 t/s | 33.6 s |
| Qwen3.6 35B-A3B | 64K | f16 | 932 t/s | 41.4 t/s | 73.5 s |
| Qwen3.6 35B-A3B | 64K | q4_0 | 750 t/s | 51.3 t/s | 90.0 s |
| Qwen3.6 35B-A3B | 128K | f16 | 617 t/s | 32.2 t/s | 216.7 s |
| Qwen3-Next 80B-A3B | 32K | f16 | 973 t/s | 46.2 t/s | 36.5 s |
| Qwen3-Next 80B-A3B | 64K | f16 | 753 t/s | 38.2 t/s | 90.5 s |
| Qwen3-Next 80B-A3B | 128K | f16 | 498 t/s | 29.1 t/s | 268.5 s |
KV-cache takeaway: q4_0/q8_0 KV improves Qwen3.6 decode speed after the context is filled, but slows prompt ingestion enough that full first-turn wall time is worse than f16 in this benchmark. Use f16 for first-turn long prompts. Consider q4_0/q8_0 only when memory pressure or long continued generation matters more than prompt-ingest speed. The 128K f16 rows completed without truncation. Raw data:
data/filled_kv_decode.csv,data/raw/2026-05-03/filled-kv-decode/, anddata/raw/2026-05-03/filled-kv-decode-128k/.
Real-corpus 64K check using this guide's own documentation files:
| Model | Prompt Type | Tokens | Prompt Eval | Decode After Fill | Wall Time |
|---|---|---|---|---|---|
| Qwen3.6 35B-A3B | synthetic repeated token | 65,533 | 932 t/s | 41.4 t/s | 73.5 s |
| Qwen3.6 35B-A3B | real guide corpus | 65,120 | 706 t/s | 40.8 t/s | 95.4 s |
| Qwen3-Next 80B-A3B | synthetic repeated token | 65,532 | 753 t/s | 38.2 t/s | 90.5 s |
| Qwen3-Next 80B-A3B | real guide corpus | 63,507 | 505 t/s | 37.8 t/s | 129.4 s |
Real-corpus takeaway: synthetic repeated-token prompts are optimistic for prompt ingest. Real documentation text slowed prompt eval by 24-33%, while decode-after-fill barely changed. Raw data:
data/raw/2026-05-03/filled-kv-decode-real-corpus/.
Based on lhl's measurements with gpt-oss-120b (tg32):
| Context | Vulkan AMDVLK | ROCm Standard | ROCm rocWMMA-tuned |
|---|---|---|---|
| 2K | 50.05 t/s | 46.56 t/s | 48.97 t/s |
| 4K | 46.11 t/s | 38.25 t/s | 45.42 t/s |
| 8K | 43.15 t/s | 32.65 t/s | 43.55 t/s |
| 16K | 38.46 t/s | 25.50 t/s | 40.91 t/s |
| 32K | 31.54 t/s | 17.82 t/s | 36.43 t/s |
Historical external result: at 32K context, standard ROCm dropped to 17.82 t/s and Vulkan held 31.54 t/s (1.8X faster), while lhl's tuned rocWMMA branch was the overall winner on that dated stack at 36.43 t/s—2X faster than standard ROCm and 15% faster than Vulkan. Upstream removal means this is not current build guidance.
Historical external evidence at 130K tokens from strixhalo.wiki is preserved below. The linked tracker was last updated 2025-08-08 and therefore predates the 2026-07-24 rocWMMA removal and ROCm 7.x/10.0; do not use it as current build guidance.
| Backend | pp512 (t/s) | tg128 (t/s) |
|---|---|---|
| Vulkan RADV | 17 | 13 |
| ROCm | 41 | 5 |
| ROCm rocWMMA-tuned | 51 | 13 |
Which backend should I use?
| You want | Use | Why |
|---|---|---|
| Easiest working chat/API path | Ollama + Vulkan/RADV (system service) | Measured 60-73 t/s class on Qwen3.6, survives restarts, vision works. |
| Fastest short-context generation | Direct llama.cpp / llama-server + Vulkan/RADV | ~100 t/s class on Qwen3-Coder 30B on official release binaries. |
| More than 8 concurrent users on MoE | Compare stock Vulkan, opt-in density Vulkan, and Lemonade ROCm on your exact model | The Vulkan 8-to-9 concurrency cliff is reproduced in MOE_CONCURRENCY.md. |
| Long context or a specific workload | The matching measured route in BEST_KNOWN_PROFILES.md |
Per-workload routes with caveats instead of stale one-line advice. |
Note: an earlier version of this diagram recommended the rocWMMA-tuned ROCm branch for long context. That route measured a 73% regression on ROCm 7.2 and failed to load current Qwen3.6 GGUFs here — see Things That DON'T Work before copying it.
Do this BEFORE installing the OS.
Navigate to Integrated Graphics then UMA Frame Buffer Size and set it to 512MB if your BIOS exposes that option. If your vendor BIOS only exposes 2GB as the minimum, leave it at 2GB; do not flash or fight the BIOS only to chase 512MB.
Why? On the primary Beelink 128GB system, the default BIOS setting reserved ~97GB for GPU VRAM and left only ~31GB visible to the OS. Setting UMA to 512MB lets Linux see almost all system RAM. Some vendor BIOSes use 2GB as the lowest fixed reserve; that is fine if Linux still sees the large system-memory pool. Vulkan/RADV uses GTT system memory, so the fixed UMA reserve is not the total memory available to the iGPU path. The practical check is
free -h: a 128GB box should show roughly 124-126GiB usable, not ~31GiB.
For a normal system, leave IOMMU Enabled or at the firmware default. This preserves NPU access, RDMA/VFIO/passthrough support, and mobile suspend behavior.
Use amd_iommu=off only as an optional profile for an always-on desktop benchmark box where the NPU and suspend do not matter. lhl's memory-bandwidth testing measured about 6% faster memory reads with IOMMU off (234 vs 221 GB/s), and this guide's primary Beelink headline runs used that profile. It is a reproducibility choice, not a universal requirement: a GMKtec EVO-X2 community Vulkan/RADV run with translated IOMMU mode reproduced the guide within about 2%.
Laptop/tablet and NPU warning: a reproduced ROG Flow Z13 case found that
amd_iommu=offprevented s0i3 hardware sleep, leaving 0% s0ix residency, spinning fans, heat, and battery drain. The Linuxamdxdnadriver also now refuses to run without IOMMU because the NPU firmware requires it. See kyuz0 issue #104 and Linux commita8878e19.
Install Ubuntu 24.04 LTS Desktop with default settings. After installation:
sudo apt update && sudo apt upgrade -yIf a specific Ubuntu 24.04 remote-desktop, screen-sharing or monitoring tool needs X11, edit GDM's daemon section. Headless inference does not require this desktop change.
sudoedit /etc/gdm3/custom.confUnder [daemon], uncomment or set WaylandEnable=false. If [daemon] is missing,
create that section and place the key immediately beneath it. Keep other sections
and settings intact; do not append the key under [debug]. Repeated edits should
leave one active key in [daemon]. Reboot deliberately after saving your work.
Ubuntu 26.04 LTS (released April 2026) ships with Linux 7.0, Mesa 26.0, and native
apt install rocm. However, 26.04 is Wayland-only (X11 switch above does not work) and the performance-relevant components (kernel, Mesa RADV) are already available on 24.04 via the kisak PPA and mainline kernel PPA. Upgrading is not needed for LLM performance. This guide stays on 24.04 LTS.
Measured setup: historical headline runs used kernel 6.19.4; the September 19 existing-user acceptance used 7.0.0-31. This is not a request to upgrade a working kernel.
- Older kernels may have gfx1151 stability or ROCm issues.
- The dated b8460/kernel 6.19.4 ROCm route used the documented
11.5.1override because that stack otherwise reported the iGPU incorrectly.- Current ROCm images that already detect native
gfx1151should run without a global HSA architecture override.- Treat kernel guidance here as this guide's tested state, not a universal support matrix.
Check your kernel:
uname -rThe following is the recorded 128GB Beelink memory profile, with a small UMA reserve. It is not a preset for 96GB systems. No automatic smaller-RAM profile is qualified here; retain distribution defaults and investigate actual runtime allocation failures before choosing host-specific limits.
Use sudoedit /etc/default/grub and preserve unrelated parameters in
GRUB_CMDLINE_LINUX_DEFAULT. For the selected 128GB profile, its relevant
parameters are:
amdgpu.gttsize=131072 ttm.pages_limit=31457280 amdgpu.cwsr_enable=0
| Parameter | Recorded scope |
|---|---|
amdgpu.gttsize=131072 |
128 GiB GTT limit; not preallocated VRAM or proof that a 128 GiB model fits |
ttm.pages_limit=31457280 |
120 GiB limit with 4 KiB pages; OS, context and runtime still need headroom |
amdgpu.cwsr_enable=0 |
Recorded compute wave save/restore setting; qualify against the selected kernel/workload |
amd_iommu=off |
Optional historical desktop profile; disables NPU access and can break deep suspend |
Leave IOMMU at its normal default for the buyer route. Preserve existing configuration and resolve conflicting parameter values before running:
sudo update-grubReboot deliberately, then compare /proc/cmdline and the live module parameters
with the chosen profile. A second setup run before reboot still has pending
boot changes. Kernel versions can change parameter support and effective limits.
Use one reviewed source for each module option. Earlier guide revisions wrote
/etc/modprobe.d/amdgpu_llm_optimized.conf with gttsize=122800, conflicting
with the GRUB value above. The script now stops for manual migration if that
legacy file exists and does not create a competing modprobe profile.
Inspect all relevant modprobe.d files, preserve administrator options, and
resolve duplicate/conflicting values deliberately. If you change module options
embedded in initramfs, run sudo update-initramfs -u -k all and require success
before rebooting. Do not infer live state from the presence of a file.
Use the distribution's device rules. Do not create world-writable (0666) KFD or DRM rules. Older revisions of this guide created /etc/udev/rules.d/99-amd-kfd.rules; review and back up that file, remove only the obsolete guide rules, and reload rules/reboot before checking permissions. Preserve any unrelated administrator rules. The script stops for manual review when that legacy file exists.
On the Ubuntu route, add your interactive user to both GPU groups, then log out and back in:
sudo usermod -aG render,video "$USER"Inspect the actual device owners, modes and service identity:
id
ls -l /dev/dri/renderD* /dev/kfd
systemctl show ollama -p User -p Group -p SupplementaryGroupsVulkan needs access to its render node; ROCm also needs KFD access. A missing KFD node does not by itself disqualify Vulkan. If Ollama runs as the ollama user, check id ollama separately: your login's groups do not grant the service access. Add only the group that owns the required device to the actual service account, then restart the service and verify GPU offload. Distribution rules and service restrictions vary; these checks are not a hardware qualification.
Choose and record one power policy per route/campaign. The script preserves the
existing policy by default; POWER_POLICY=tuned bash setup.sh selects the older
accelerator-performance reproduction profile and stops if a conflicting daemon
is active. The commands below are for deliberately selecting that profile, not a
universal benchmark prerequisite. Controlled desktop-performance/DPM-auto runs
are also retained in Reproducibility.
sudo apt install tuned -y
sudo systemctl disable --now power-profiles-daemon
sudo systemctl enable --now tuned
sudo tuned-adm profile accelerator-performanceVerify:
tuned-adm active
# Expected: Current active profile: accelerator-performance
systemctl is-active power-profiles-daemon
# Expected: inactivePower-policy effects depend on the workload and stack. The historical combined driver/policy observations do not isolate a universal tuned percentage. The 221/234 GB/s external comparison concerns the IOMMU experiment, not a matched tuned-only test.
Avoid competing power-policy managers. For a tuned reproduction, verify tuned is active and power-profiles-daemon inactive. For a different recorded policy, verify that policy instead; do not alter it merely to pass an older checklist.
The default Mesa on Ubuntu 24.04 is significantly slower. Upgrade to Mesa 26.0.2 or newer from the kisak-mesa PPA. Exact driver metadata is recorded per run in the CSV and raw evidence when available.
sudo add-apt-repository ppa:kisak/kisak-mesa
sudo apt update
sudo apt upgrade -yVerify:
vulkaninfo --summary 2>&1 | grep driverInfo
# Expected: Mesa 26.0.2+ RADV from kisak-mesa PPA.Impact: Mesa 25.2.8 to 26.0.1 gave +9% prompt eval (87 to 96 t/s). Mesa 26.0.1 to 26.0.2 gave an additional small improvement.
Note: Investigate DKMS failures before treating an upgrade as complete; see Troubleshooting.
Record clocks under the actual workload. A 2900 MHz target belongs to the older high-DPM desktop profile, not every model, OEM or idle state:
cat /sys/class/drm/card*/device/pp_dpm_sclk
# Asterisk identifies the currently selected state; compare with the recorded profile.A historical stack was observed stuck at 900 MHz. Low clocks can also reflect idle/light load, thermal or power policy; inspect utilization and the selected profile before diagnosing a fault.
dpkg -l | grep linux-firmware | head -5Firmware compatibility:
linux-firmware-20251125has a documented Strix Halo ROCm failure history (toolbox source). If symptoms began after installing it, inspect the exact package version and your distribution's available versions. The guide's historical working Ubuntu package is recorded in Reproducibility; it is not a universal firmware guarantee.dpkg-query -W linux-firmware apt-cache policy linux-firmwareSelect an unaffected package available for your distribution, install that exact version using its package manager, then reboot and repeat the GPU/model checks. A version from another distribution is not a safe substitute.
apt-mark holdonly prevents package changes: it does not downgrade firmware. If you temporarily hold a restored working package, record the reason and revisit the hold when a suitable update is verified.
Ollama is the easiest way to run LLMs on Strix Halo. With the right configuration, it works great.
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.31.2 shVersion 0.31.2 is the reboot-qualified fresh-install baseline. The script retains existing runtimes; verify their actual running version separately. Newer-model routes, including Qwen3.8, have their own measured runtime requirements and are explicit opt-ins rather than an automatic baseline upgrade.
Historical update (April 2026): The measured Ollama 0.20.4/kernel 6.19.x ROCm route used
HSA_OVERRIDE_GFX_VERSION=11.5.1(ollama/ollama#14855). That row measured 42.4 t/s versus 46.6 t/s on Vulkan for Qwen3.5-35B. Current native-gfx1151ROCm builds should be tested without a global HSA override; Vulkan remains the easier measured beginner path.
sudo systemctl edit ollamaPreserve existing settings and add the following between the comment lines. Check for existing assignments first; resolve conflicting values deliberately. The automated script uses a separate 60-strix-halo-guide.conf and refuses conflicts instead of replacing your override.conf.
[Service]
Environment="OLLAMA_VULKAN=1"
Environment="OLLAMA_IGPU_ENABLE=1"
Environment="HIP_VISIBLE_DEVICES=-1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="AMD_VULKAN_ICD=RADV"
Environment="VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json"
Environment="OLLAMA_NUM_PARALLEL=1"Reload and inspect the effective environment before restarting. Later drop-ins can override values; EnvironmentFile and UnsetEnvironment can change the final environment too. Review those manually if present. Do not paste service environment output publicly: it can contain credentials.
sudo systemctl daemon-reload
systemctl show ollama -p Environment -p EnvironmentFiles -p UnsetEnvironment
sudo systemctl restart ollama| Variable | Purpose |
|---|---|
OLLAMA_VULKAN=1 |
Select the measured Vulkan route; backend performance depends on workload |
OLLAMA_IGPU_ENABLE=1 |
Let current Ollama builds use the Strix Halo integrated GPU instead of dropping it during GPU discovery |
HIP_VISIBLE_DEVICES=-1 |
Disable HIP device enumeration (avoids ROCm fallback) |
OLLAMA_FLASH_ATTENTION=1 |
Enable flash attention on the selected runtime; verify workload behavior |
OLLAMA_CONTEXT_LENGTH=8192 |
Limit context to prevent OOM (increase if needed) |
AMD_VULKAN_ICD=RADV |
Force RADV driver (faster than AMDVLK for general use) |
VK_ICD_FILENAMES=... |
Explicitly point to RADV ICD file |
OLLAMA_NUM_PARALLEL=1 |
Single request at a time (maximizes single-request speed) |
# Fast MoE model, great for general use and coding (~20GB)
ollama pull qwen3.6:35b-a3b
# Current official dense multimodal model, measured route in this guide (~18GB)
ollama pull qwen3.8:27b
# Higher quality MoE, Q8_0 quantization (~32GB)
ollama pull qwen3-coder:30b-a3b-q8_0
# Google's MoE model, strong reasoning (~16GB)
ollama pull gemma4:26b-a4b
# Large coding MoE (80B total, 3B active) for complex tasks (~51GB)
ollama pull qwen3-coder-nextollama run qwen3.6:35b-a3bVerify a useful text response and inspect ollama ps plus runtime logs for GPU offload. Speed alone does not establish correct setup; match the exact model/runtime/profile before comparing rates.
From a local checkout of this guide:
bash scripts/ollama_smoke.sh
bash scripts/ollama_smoke.sh qwen3-coder-next 'Explain backpropagation in simple terms.'The setup script also installs the same helper as ~/bench-ollama.sh. It serializes
quoted/multiline prompts, bounds HTTP duration, requests up to 128 output tokens,
and rejects HTTP errors, empty/incomplete responses and invalid timing counters.
Natural completion can be shorter. This is a smoke check, not a controlled
throughput comparison: inspect ollama ps and runtime logs to verify actual GPU
offload, then use the reproducibility protocol.
bash scripts/ollama_smoke.sh qwen3.6:35b-a3b \
'Review a Python web application architecture covering sessions, database access, caching, background jobs and error handling. Explain the tradeoffs with concrete examples.'Record the actual input/output token counts, cache policy, warmup and repetitions before comparing rates. A longer instruction is not by itself a long-context test.
Prompt processing speed scales with prompt length due to GPU parallelism:
| Prompt Tokens | pp (qwen3.5:35b-a3b) | pp (qwen3-coder-next) |
|---|---|---|
| 12-14 | 121 t/s | 91 t/s |
| 21-23 | 182 t/s | 130 t/s |
| 120-122 | 457 t/s | 301 t/s |
For ROCm-specific workloads, batch processing, and long-context experiments, use llama.cpp with ROCm via kyuz0 containers. For short-context MoE inference, current measured results still favor Vulkan RADV.
Current rule: start with no global
HSA_OVERRIDE_GFX_VERSIONand verify thatllama-cli --list-devicesreportsgfx1151. A 2026-07-29 Beelink / Ubuntu validation found that a stale hostHSA_OVERRIDE_GFX_VERSION=11.0.0was inherited by Distrobox, changed detection togfx1100, and crashed current ROCm 7.2.4. Removing it restored nativegfx1151detection and inference. The older11.5.1workaround is retained only for reproducing the dated b8460/kernel 6.19.4 rows.
sudo apt install podman -y
curl -s https://raw.githubusercontent.com/89luca89/distrobox/main/install | sudo shNote: Ubuntu 24.04 does not include
toolboxin its repos. Use Distrobox instead. The defaulttoolboxon Ubuntu also breaks GPU access.
The rocm-7.2 command below is a dated reproduction route, not a floating latest
recommendation. The upstream toolbox now documents a rocm-10.0 stable route;
use its current instructions and pin an image digest for a new qualification
campaign. This guide has not promoted that available image to its measured default.
distrobox create llama-rocm-72 \
--image docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.2 \
--additional-flags "--device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"distrobox enter llama-rocm-72
rocm-smi # Should show your gfx1151 GPUThe following is a historical reproduction command, not a v0.4.1 example.
The rocm-7.2 tag alone does not pin a binary: match the archived build/digest
and inspect llama-bench --help before using its old -mmp flag. For a new
candidate, use the explicitly versioned build example instead.
export ROCBLAS_USE_HIPBLASLT=1
llama-bench -m ~/models/your-model.gguf -fa 1 -ngl 999 -mmp 0 -p 128,512 -n 128Critical flags:
| Flag | Impact | Notes |
|---|---|---|
-fa 1 |
Flash attention | Recorded profile; validate model/build support |
-mmp 0 (--no-mmap) |
Disable mmap | Recorded route; compare load/memory behavior for the selected stack |
ROCBLAS_USE_HIPBLASLT=1 |
Historical environment setting | Effect depends on the runtime path; no universal improvement established |
-ngl 999 |
Full GPU offload | Use all available VRAM |
The kyuz0 pre-built binary includes the critical compiler flag
--amdgpu-unroll-threshold-local=600which works around the LLVM compiler regression in ROCm 7+. Self-compiled binaries without this flag may be significantly slower.
Pinned v0.4.1 ROCm/HIP route: scoped Coder/Gemma controls passed, not all-model correctness. See the exact test limits. Use a separate checkout/build; retain the known control. See the HIP correctness queue before promotion.
# Inside a ROCm container
git clone --branch v0.4.1 --depth 1 https://github.com/ggml-org/llama.cpp llama.cpp-v0.4.1-hip
cd llama.cpp-v0.4.1-hip
# Current HIP build
cmake -B build -S . \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS="gfx1151" \
-DCMAKE_HIP_FLAGS="-mllvm --amdgpu-unroll-threshold-local=600" \
-DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)Upstream change (2026-08-30 review): do not pass
GGML_HIP_ROCWMMA_FATTN.llama.cppPR #26046, merged 2026-07-24, removed the rocWMMA FlashAttention implementation; HIP builds now use the newer MMA kernel. The older local and external rocWMMA material inROCM_ROCWMMA_BASELINE.mdis historical evidence only.
kyuz0's vLLM toolboxes enable API serving on gfx1151. Treat vLLM as a separate serving benchmark path, not as something to install into the host Python environment.
The command below preserves the May :stable reproduction route. Upstream now
uses :latest for its verified vLLM route and :dev for development. Those moving
tags are not immutable benchmark identities: select and record a digest, then
qualify the exact image before treating it as a new recommendation.
Local preflight status: vllm-gfx1151 was created and smoke-tested on 2026-05-03 with the :stable image. See VLLM_BASELINE.md. This is setup evidence, not a throughput benchmark.
distrobox create vllm-gfx1151 \
--image docker.io/kyuz0/vllm-therock-gfx1151:stable \
--additional-flags "--device /dev/kfd --device /dev/dri --group-add video --group-add render --security-opt seccomp=unconfined"
distrobox enter vllm-gfx1151
rocm-smi
start-vllmRecord vLLM results separately from llama.cpp server results. At minimum, capture image tag, ROCm/TheRock build, model, quant, max context, concurrency, aggregate throughput, TTFT, p50/p95 latency, memory use, and any kernel compile/cache warmup behavior.
Known vLLM issues on gfx1151:
- Qwen3.5 block_size validation (issue #28): Hybrid mamba/attention models compute
block_size=1056which gets rejected by a hardcoded whitelist. Fix available in the issue. - MIOpen encoder hang (issue #30): Vision models hang during kernel search because MIOpen lacks pre-compiled solver DBs for gfx1151. Workaround: disable encoder profiling.
Tested models on vLLM:
| Model | Max Context |
|---|---|
| Llama-3.1-8B | 128K |
| Gemma-3-12b | 128K |
| Qwen3-Coder-30B-A3B (GPTQ 4-bit) | 256K |
| gpt-oss-120b | 128K |
| Qwen3-Next-80B-A3B (GPTQ Int4) | 256K |
For models that exceed 128GB, you can cluster multiple Strix Halo machines using RDMA.
From kyuz0's vLLM clustering guide:
Hardware needed:
- 2x Strix Halo machines (e.g., Framework Desktop)
- 2x Intel E810-CQDA1 100GbE NICs
- 1x DAC cable (direct attach copper, no switch needed for 2 nodes)
Performance:
- ~50 Gbps bandwidth, ~5 us latency (vs ~70-100 us TCP/IP)
- TP=2 across two 128GB machines provides 256GB aggregate physical RAM, with per-node runtime/headroom constraints; it is not one universally shared allocation pool.
- This two-node RDMA/vLLM route is separate from AMD's February 25 demonstration: four Framework 128GB nodes, llama.cpp RPC, a 375GB Kimi K2.5 UD-Q2_K_XL artifact and 5Gbps Ethernet. That external capacity demonstration does not qualify trillion-parameter inference on the two-node recipe.
Additional kernel parameter for clustering:
pci=realloc
Network configuration:
# Set MTU to 9000 (jumbo frames)
sudo ip link set <interface> mtu 9000sudo apt install openssh-server fail2ban -ysudo sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sudo systemctl restart sshfail2ban starts automatically and blocks IPs after repeated failed login attempts. We found 68 brute-force attempts on our system within hours of enabling SSH -- fail2ban is essential.
We tested both Vulkan drivers via llama-bench. Results depend heavily on the llama.cpp build version:
With kyuz0 containers (b8298):
| Driver | Model | pp512 | tg128 |
|---|---|---|---|
| RADV | Qwen3.5-35B-A3B | 859 | 52.15 |
| AMDVLK | Qwen3.5-35B-A3B | 576 | 55.54 |
| RADV | Llama 2 7B | 1377 | 48.12 |
| AMDVLK | Llama 2 7B | 327 | 48.02 |
With newer tested llama.cpp (b8460) -- AMDVLK advantage is gone:
| Driver | Model | pp512 | tg128 |
|---|---|---|---|
| RADV | Qwen3.5-35B-A3B | 1080 | 64.85 |
| AMDVLK | Qwen3.5-35B-A3B | 663 | 64.10 |
AMDVLK is discontinued. Uninstall it -- even inactive, its ICD file silently hijacks Vulkan and halves your pp speed. See AMDVLK warning above.
Our recommendation: Use RADV. AMDVLK is discontinued (last release April 2025). In the recorded newer Vulkan driver comparison, RADV won both pp and tg. AMDVLK's reported 2 GiB buffer limit is a possible contributor to the 3-4X slower pp on the dense-model pair, not an isolated causal result. The measured beginner route uses RADV.
Optimal ubatch sizes per driver (from lhl's testing):
- AMDVLK:
-ub 512 - RADV:
-ub 1024 - ROCm HIP:
-ub 2048
These findings correct several common recommendations found in other Strix Halo guides.
| Issue | Common Advice | Reality | What Happens If You Try |
|---|---|---|---|
| Ollama HIP/ROCm | "Always add an HSA architecture override" | The dated Ollama 0.20.4/kernel 6.19 route needed 11.5.1; current native-gfx1151 builds should start without a global override |
Verify the detected architecture and keep Vulkan as the easier measured beginner path |
iommu=pt for speed alone |
"Use pass-through only for more speed" | No benefit over default in lhl's memory-read test | Still useful when IOMMU-dependent NPU, suspend, RDMA, VFIO, or passthrough behavior matters |
| AMDVLK for all workloads | "AMDVLK is fastest" | Project discontinued (last release April 2025). RADV beats AMDVLK on both pp (+63%) and tg. Worse: even if you don't use AMDVLK, its ICD file (/etc/vulkan/icd.d/amd_icd64.json) silently hijacks Vulkan and halves your pp speed. You won't see an error -- just mysteriously slow prompt processing |
Uninstall it completely: sudo dpkg -r amdvlk && sudo rm -f /etc/vulkan/icd.d/amd_icd64.json. Verify with llama-bench: RADV shows (RADV STRIX_HALO) with shared memory: 65536. AMDVLK shows (AMD open-source driver) with shared memory: 32768 |
| rocWMMA on upstream llama.cpp | "Enable for 2x speed" | 73% regression on ROCm 7.2 | Massively slower prompt processing |
| BIOS VRAM increase for speed | "More GPU VRAM = faster" | Zero speed difference, but a very large fixed UMA reserve can cripple OS-visible RAM and GTT capacity. Use 512MB if available; 2GB is fine when that is the vendor minimum. | If Linux only sees ~31GB on a 128GB box, large models will not load |
| ROCm 7.0 RC | "Use ROCm 7 RC" | Segfaults on kernel 6.18.14+ | HSA_STATUS_ERROR crash |
| Reusing old HSA overrides on a current ROCm image | "Keep the workaround forever" | A stale host 11.0.0 override was inherited by Distrobox and changed native gfx1151 to gfx1100 |
Current ROCm 7.2.4 crashed in libamdhip64; unsetting the obsolete override restored inference |
| linux-firmware-20251125 | Auto-update | Breaks ROCm on Strix Halo | Instability, crashes |
| PyTorch / HuggingFace Transformers | "Any model/runtime combination will be fast" | A reported 70B decode workload spent 92-95% of decode time in hipMemcpy and reported ~1.5 t/s versus 4.8 t/s for its llama.cpp comparison | Workload-specific transfer bottleneck, not a verdict on all PyTorch UMA support. The guide separately documents scoped Unsloth and vLLM successes; qualify the exact workload |
| Optimization | Impact | How |
|---|---|---|
| Mesa 25.2.8 to 26.0.2 | +9-10% pp | sudo add-apt-repository ppa:kisak/kisak-mesa |
| Flash Attention | +13% pp | -fa 1 or OLLAMA_FLASH_ATTENTION=1 |
| Load policy | Historical +22% pp128 on the recorded stack | Not a universal optimum. Old builds used -mmp 0; v0.4.1 uses --load-mode. Compare load time, memory and correctness on the exact build |
| hipBLASLt | +8% tg | ROCBLAS_USE_HIPBLASLT=1 (ROCm only) |
| tuned accelerator-performance | Historical reproduction profile; no isolated universal gain established | Optional; preserve or record the selected power policy and avoid competing managers |
| RADV over AMDVLK | +63% pp, +1.2% tg | Uninstall AMDVLK entirely (see above). AMD_VULKAN_ICD=RADV works too but is easy to forget |
OLLAMA_IGPU_ENABLE=1 |
Avoids CPU-only Ollama on current builds | Required for both the Ollama 0.31.1 local-binary comparator and the normal 0.31.2 service path on the measured Beelink system |
amd_iommu=off |
About +6% memory reads in one measured desktop test | Optional benchmark profile; do not use for NPU or mobile suspend workflows |
| BIOS UMA/VRAM reserve low enough | OS sees ~124-126GiB instead of ~31GiB on 128GB systems; GTT gets the large shared pool | No speed change from 512MB vs sane low reserves, but required to avoid losing most system RAM. Use 512MB if available; 2GB is fine when that is the vendor minimum |
HIP_VISIBLE_DEVICES=-1 |
Fixes Ollama crash | Required for Vulkan-only mode |
| LLVM unroll workaround | Restores ROCm 7+ perf | -mllvm --amdgpu-unroll-threshold-local=600 |
| lhl's rocWMMA-tuned | Historical external 2X tg at 32K context | Removed upstream path; preserve as dated evidence, not current build guidance |
| Runtime version | The historical b8298-to-b8460 MoE comparison improved substantially | Preserve that dated result; test newer pinned builds separately rather than promising an upgrade gain |
No global HSA architecture override on current native-gfx1151 builds |
Avoids silently forcing the wrong target | Verify gfx1151; use 11.5.1 only to reproduce the dated b8460/kernel 6.19.4 route |
Before changing kernels, ROCm containers, Ollama deployment type, or DeepSeek environment variables, check the scoped ROCm and vLLM bugwatch. It currently tracks:
- one kernel-specific ComfyUI/FLUX load deadlock report;
- a reported ROCm
mmaplimit above 64GB on one Ubuntu 26.04 setup; - a container-only Ollama UMA-reporting regression;
- a Windows bare-metal
llama.cppROCm performance regression; - a DeepSeek ROCm environment-variable trap where defining a variable as
0can still enable its path; and - an older ROCm playbook failure that AMD reports as passing with current pinned wheels.
These are troubleshooting signals, not proof that every Strix Halo system or every version is affected. Match the exact OS, kernel, backend, deployment type, and artifact before applying a workaround.
Fail-Safe's three-system Corsair AI Workstation 300 campaign found that two systems had previously booted without the out-of-tree ec_su_axb35 module after a kernel update, causing dependent custom fan/power services to fail. That is a plausible major contributor to the earlier sustained-load event, not a proven sole root cause and not evidence that every Corsair or Strix Halo system needs a clock cap.
If you use this custom fan path, verify the module and services after kernel updates before sustained inference. The contributor's measured 2400 MHz tradeoff is scoped to the tested fleet. See THERMAL_STABILITY.md for commands, charts, raw evidence, limitations, and the merged upstream fan-reset source change and installed-version qualification gap.
Symptoms on that dated stack: Without the workaround, ROCm containers
segfaulted. ggml_cuda_init reported gfx1100 (0x1100) instead of
gfx1151.
Dated fix: These variables were used for the measured b8460/kernel 6.19.4 route:
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export HSA_ENABLE_SDMA=0With this fix, ROCm worked on that measured setup and improved prompt processing versus the older 6.18.14 row. It is historical reproduction metadata, not the current default.
Current ROCm builds with native Strix Halo support should report gfx1151.
Before debugging a current container, check the host and container:
printenv HSA_OVERRIDE_GFX_VERSION
llama-cli --list-devicesIf an old override is set, remove it from host shell/service startup files and retry with:
env -u HSA_OVERRIDE_GFX_VERSION llama-cli --list-devicesThe Kyuz0 toolbox cross-OEM validation contains the exact Beelink/Ubuntu failure, kernel log, corrected repeats, image digests, and minimal rootless Podman commands.
Qwen3.5 ROCm Hang Bug (ROCm #6027)
Symptoms: Qwen3.5 models (35B-A3B and 27B) hang during load_tensors on ROCm. CPU pegs at 99.9%.
Historical status: the cited report is closed. The submitted conservative one-layer attempt also hung; closure does not establish that this guide's exact model/build is now qualified.
The reported --n-gpu-layers 1 attempt is a preserved failure, not a workaround. Match the model, artifact, runtime and logs against a known working route before retrying; do not infer success from newer availability alone.
Symptoms: GPU stays at 900 MHz instead of 2900 MHz, causing ~8% performance loss.
Check:
cat /sys/class/drm/card*/device/pp_dpm_sclk
# Should show: 2: 2900Mhz *For a deliberate reproduction of the older high-DPM desktop profile only, record the existing state before a temporary change, then restore that exact state after the test. First identify the intended GPU device; do not write through a wildcard on a multi-GPU system.
# Read current state for the GPU you identified; change only as part of a selected profile.
cat /sys/class/drm/card*/device/power_dpm_force_performance_levelNewer kernels (6.18.4+) recognize gfx1151's 1.5X VGPR capacity compared to standard gfx11 chips. This enables better occupancy for compute shaders. If you're on an older kernel, you may not be getting full performance.
DKMS mt7925 WiFi Errors During apt install
You'll see this on every apt install:
Error! Bad return status for module build on kernel: 6.18.14-061814-generic
dkms autoinstall failed for mt76-mt7925(10)
The original system could use its in-kernel WiFi driver despite this DKMS failure. Do not assume that applies to your adapter/kernel, or that the displayed module version matches the installed package. Inspect the actual state first:
dkms status
lspci -kPrompt Processing Slower Than Expected (llama.cpp -ub Silently Clamped)
If prompt processing (pp512) is well below the guide's figures while generation looks normal, check your batch flags: -ub larger than -b is silently clamped to min(n_batch, n_ubatch) with no warning or error. A community GTR9 Pro report measured pp512 733.90 with -b 256 -ub 1024 versus 945.83 with -b 2048 -ub 512 (+28.9%), generation unchanged. Also check Mesa: stock Ubuntu 24.04 Mesa 25.2.8 versus kisak 26.1.7 measured a further +16.9% pp512 on the same benchmark, and no reboot is needed -- restarting llama-server picks up the new userspace RADV driver. See COMMUNITY_RESULTS.md.
Ollama Runs But Is Very Slow (Silent CPU-Only Fallback)
If generation works but speed is far below the numbers in this guide (for example single-digit t/s on a 30B MoE model), Ollama has likely dropped the integrated GPU and is running CPU-only. Measured 0.31.x builds can detect the Radeon 8060S and still drop the iGPU path when OLLAMA_IGPU_ENABLE=1 is missing.
# Check whether the model is actually on GPU
ollama ps
journalctl -u ollama --no-pager | grep -iE "vulkan|igpu|gpu" | tail -20
# Use the complete service block in Step 5.2 if this matches the selected route.Ollama "Out of Memory" Even with Small Models
Unexpected backend selection was one historical cause on the measured Ollama stack. OOM can also reflect model size, context/KV allocation, concurrency, available memory or a runtime defect. Inspect the exact version, selected backend and logs before changing configuration:
# Check current Ollama environment
systemctl show ollama | grep Environment
ollama ps
free -h
journalctl -u ollama -n 50 --no-pagerIf the logs establish that the intended Vulkan profile is missing, use the
complete [Service] / Environment= block in Step 5.2,
preserving unrelated settings and checking effective conflicts before restart.
ROCm container reports gfx1100 or segfaults
First check whether a legacy host override is forcing the wrong architecture:
printenv HSA_OVERRIDE_GFX_VERSION
env -u HSA_OVERRIDE_GFX_VERSION llama-cli --list-devicesOn current native-gfx1151 stacks, keep the override unset. A stale
HSA_OVERRIDE_GFX_VERSION=11.0.0 caused current ROCm 7.2.4 to report
gfx1100 and crash during model load on the measured Beelink/Distrobox
route. Only use 11.5.1 when deliberately reproducing the older
b8460/kernel 6.19.4 evidence above.
Verifying GPU Memory Configuration
# Check TTM pages limit
cat /sys/module/ttm/parameters/pages_limit
# Check GTT size
cat /sys/module/amdgpu/parameters/gttsize
# Check Vulkan driver
vulkaninfo --summary 2>&1 | grep -E "driverName|driverInfo"
# Check OS-visible RAM
free -h
# Check GPU memory allocation
for file in /sys/class/drm/card*/device/mem_info*; do
echo "$file: $(cat $file)"
donerocm-smi Shows Wrong VRAM
For APUs with unified memory, a small mem_info_vram_total can reflect the fixed
UMA reserve. GTT limits depend on RAM, kernel and the selected profile; ~128 GiB
is not a universal expectation or a promise that a model of that size fits.
tuned Not Running After Reboot
Apply this only if you selected the tuned reproduction profile. A different recorded desktop-performance policy can deliberately leave tuned inactive.
# Check status
tuned-adm active
# If not running:
sudo systemctl disable --now power-profiles-daemon
sudo systemctl enable --now tuned
sudo tuned-adm profile accelerator-performance
# Verify it persists
tuned-adm active
systemctl is-active power-profiles-daemonGPU Stuck at Low Clock Speed
# Check current clock
cat /sys/class/drm/card*/device/pp_dpm_sclk
# Compare under load with the selected campaign's power policy.
# Do not make a temporary high-DPM diagnostic persistent by default.Historical compatibility snapshot from the March/May test campaign:
| Kernel | ROCm 6.4.4 | ROCm 7.2 | ROCm 7 Nightly | Vulkan (Ollama) |
|---|---|---|---|---|
| 6.17.7 | Works (with right firmware) | Unknown | Works | Works |
| 6.18.4-6.18.14 | Works (patched) | Works | Works | Works |
| 6.19.4 | Works (HSA fix) | Works (HSA fix) | Unknown | Works |
As of 2026-08-30, ROCm Core SDK 10.0.0
is released and lists gfx1151 in its official supported-hardware table.
Nothing in this guide has installed or qualified ROCm 10.0; this historical
matrix and the measured ROCm 7.2/7.14 evidence remain unchanged.
Rules for interpreting this dated matrix:
- Kernel 6.18.4+ changed gfx1151 handling; use current ROCm builds/containers instead of old ROCm RC builds
- The measured b8460/kernel 6.19.4 route used
HSA_OVERRIDE_GFX_VERSION=11.5.1 - The documented linux-firmware-20251125 failure applies to the recorded stack; investigate exact package/runtime combinations
- A later date alone does not qualify firmware; preserve known-working package metadata and retest after changes
Current measured recommendation: use a current ROCm build with native
gfx1151support and no global HSA architecture override. Keep the older variables only in commands that reproduce the dated rows. Before publishing benchmark numbers, also verify kernel, ROCm version, Mesa, AMDVLK removal, GPU clock, andtunedstatus.
Beelink wall-power efficiency is not published yet. powercap is empty on this system, but amdgpu exposes PPT telemetry through power1_average / power1_input. A 2026-05-16 local PPT run measured roughly 111-113 W during Qwen3-Coder/Qwen3.6 Vulkan workloads, but this is GPU/APU telemetry, not wall power.
Community wall-power data does exist from Corsair AI Workstation 300 systems. The current issue #6 cross-section measures about 150 W / 1.6 J/token for Qwen3-Coder, 148 W / 2.0 J/token for Qwen3.6, 174 W / 3.1 J/token for gpt-oss-120b, and 137 W / 3.4 J/token for Qwen3-Coder-Next during sustained generation. Use that as practical community context, not as a Beelink claim.
A separate three-system Corsair thermal/SCLK campaign reports Linux AMDGPU socket-power telemetry from 80.90 W at a 2200 MHz cap to 118.71 W at 2600 MHz, plus 119.54 W mean in bounded stock controls. Those are socket-power readings, not wall power or Beelink measurements. See THERMAL_STABILITY.md.
See POWER_BASELINE.md, COMMUNITY_RESULTS.md#whole-system-power, data/community_power.csv, data/beelink_power_telemetry.csv, and scripts/sample_power.py before adding tokens-per-watt claims.
After completing setup, verify each item:
-
free -hshows most of your installed memory, not ~31GB (~124GiB on 128GB systems; lower on 96GB systems) -
vulkaninfo --summaryshows the intended RADV driver; September 19 acceptance used Mesa 26.2.3, while historical benchmark rows retain their own driver metadata. Do not upgrade solely to match this checklist. - Record the selected power policy and verify its active manager; tuned is required only for a tuned reproduction
- Check GPU clocks/utilization under the actual workload against that profile, not an idle universal 2900 MHz threshold
- Compare live GTT/TTM values with the selected RAM/kernel profile; 31457280 pages is the recorded 128GB-system limit, not a 96GB preset
-
ollama --versionreturns without error - A useful text task succeeds;
ollama psand runtime logs establish actual GPU offload independently of speed -
systemctl show ollama | grep EnvironmentincludesOLLAMA_VULKAN=1andOLLAMA_IGPU_ENABLE=1 -
cat /etc/default/grub | grep CMDLINEincludes the GTT/TTM parameters;amd_iommu=offappears only if you deliberately selected the optional desktop benchmark profile - Record the exact kernel/runtime pair; current native gfx1151 ROCm starts without a global HSA override, while dated reproduction commands retain theirs
- Record the exact firmware package and verify the selected runtime after any change; version dates alone do not prove compatibility
- kyuz0/amd-strix-halo-toolboxes -- Community standard containers for llama.cpp (1.2k+ stars)
- kyuz0/amd-strix-halo-vllm-toolboxes -- vLLM serving + RDMA clustering
- kyuz0/amd-strix-halo-gfx1151-toolboxes -- Meta repository with all toolboxes
- kyuz0 Backend Benchmarks Dashboard -- Interactive benchmark comparison
- lhl/strix-halo-testing -- Deep performance research and rocWMMA patches
- nabe2030/hip-vs-vulkan-evo-x2 -- Independent HIP versus Vulkan workload-crossover benchmark on Strix Halo
- hec-ovi/vllm-awq4-qwen -- Experimental Qwen3.6 AWQ/DFlash vLLM path for Strix Halo
- strixhalo.wiki and llm-tracker.info -- historical community references last updated 2025-08-08; they predate the 2026-07-24 rocWMMA removal and ROCm 7.x/10.0, so do not follow their ROCm 6.5-nightly or rocWMMA advice as current guidance
- Level1Techs Forum -- Community benchmark results
- Framework Community -- Framework Desktop discussions
- ROCm Strix Halo Optimization Guide -- Official AMD guide
Use the Strix Halo model hub for a web view that separates models measured by this guide from artifacts verified to exist but not measured here, plus dated 128GB published-size fit tiers. The tables below remain the canonical in-repository setup guidance.
Not sure which model to run? Here's what we recommend based on use case. Recommendations reviewed 2026-08-30; wording corrected September 5. Speeds belong to different measured guide rows, not a matched quality evaluation. Advertised context is not a locally qualified context limit.
| I want to... | Model | Size | Speed | Why |
|---|---|---|---|---|
| Fastest 30B-class Qwen direct row | Qwen3-30B-A3B-Instruct-2507 (IQ4_XS) | 13.9 GB | 100.0 t/s | First direct llama-bench row above 100 t/s; general-instruct, not coding-specific |
| Code (best speed) | Qwen3-Coder 30B-A3B (Q4_K_S) | 17.5 GB | 101.0 t/s | Fastest measured coding speed on official b9851 Vulkan; speed-first quant |
| Code (balanced quant) | Qwen3-Coder 30B-A3B (UD-Q4_K_XL) | 17.7 GB | 96-97 t/s | Strong coding default, MoE architecture |
| Code (higher-bit quant) | Qwen3-Coder 30B-A3B (Q8_0) | 32 GB | 51 t/s | Same model, higher fidelity quantization |
| Chat (general) | Qwen3.6 35B-A3B (Q4_K_M) | 20 GB | 63 t/s | Measured general-chat starting point; task quality depends on the workload |
| Chat (no thinking) | Qwen3.6 35B-A3B (no-think) | 20 GB | 63 t/s | Same speed, direct answers |
| Code (80B MoE; advertised 256K ctx) | Qwen3-Next 80B-A3B | 42.90 GiB | 59 t/s | 80B MoE, only 3B active, 256K context |
| Coding-model experiment | Qwen3-Coder-Next | 51 GB | 38 t/s | 80B-A3B coding MoE; no local quality ranking established |
| Reasoning / current Google route | Gemma 4 26B-A4B IT QAT | 14.2 GB | 74.8 t/s direct; 102.7-110.0 t/s MTP server | Strong current Google-model route. Use the direct row for benchmark comparisons; use MTP only for server/speculative experiments |
| Analyze images | Qwen3.8 27B (qwen3.8:27b) |
~18 GB | 20.4 t/s | Current official dense multimodal route, measured here with image, tools, and thinking |
| Largest direct GGUF capacity tested | DeepSeek V4 Flash 284B-A13B (UD-IQ2_XXS) | 90.9 GB | 13.3 t/s | Load and basic arithmetic passed; low-bit quality and real-task usefulness remain unqualified |
| "Can it run?" | Llama 4 Scout 109B | 61 GB | 18 t/s | 109B artifact on one 128GB machine; exceeds 24GB VRAM without offload |
| Process documents | Qwen3.6 35B-A3B (Q4_K_M) | 20 GB | 63 t/s | Fast enough for RAG pipelines |
| Learn / experiment | LFM2.5 8B-A1B (Q4_K_M) | ~5 GB | 169-176 t/s | Small current MoE, fastest practical-model rows measured here |
| Throughput testing | Qwen3-0.6B (Q8_0) | 0.8 GB | 266 t/s | Speed ceiling benchmark |
How to install any model:
# Via Ollama (easiest)
ollama pull qwen3.6:35b-a3b
# For llama-bench direct (need GGUF file)
# Download from huggingface.co, place in ~/models/ Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf
| | | | | | |
| | | | | | +-- Quantization (see Glossary)
| | | | | +-- "Unsloth Dynamic" quant method
| | | | +-- Fine-tuned for instructions
| | | +-- 3B Active parameters (MoE)
| | +-- 30B Total parameters
| +-- Optimized for coding
+-- Model family (by Alibaba)
Compare the exact task, quality requirement and actual bill. Local throughput does not establish equivalence with a hosted model, and buying hardware does not remove all software, power, maintenance or time costs.
Illustrative token-fee scenario, checked September 13, 2026: Claude Sonnet 5 standard pricing is $2 per million input tokens and $10 per million output tokens. Ten million uncached input tokens plus one million output tokens therefore costs $30 in base token fees. This excludes tools, cache writes/reads, batch discounts, regional premiums and tax. It is not a claim that a local Qwen model performs the same work at the same quality. Anthropic pricing.
For your own scenario:
API cost = input_tokens / 1e6 × input_tariff
+ output_tokens / 1e6 × output_tariff + cache/tools/other fees
Electricity = days × (active_W × active_hours + idle_W × idle_hours) / 1000 × price_per_kWh
Monthly ownership = purchase_cost / assumed_lifetime_months + electricity + maintenance
For example, assumed, unmeasured 120 W active for one hour/day plus 30 W idle for 23 hours/day costs $3.65 over 30 days at $0.15/kWh. At 140 W active it is $3.74. These duty cycles are explicit assumptions, not inferred from token counts. The dated $4,349 Beelink example amortized over an assumed 60 months adds $72.48 per month before maintenance/resale. Beelink wall power remains unmeasured here; amdgpu PPT is not a wall-meter substitute.
Break-even exists only when verified avoided spending exceeds incremental local costs, and hardware remains useful long enough. No payback period or subscription replacement is established by this guide.
Local inference can keep prompts on the machine after artifacts are provisioned, provided models, embeddings, tools and integrations also run locally. Optional cloud, web-search and remote-tool features change that boundary. Capacity and latency still limit requests, and refusal/filter behavior depends on the selected weights and application. Review Ollama's cloud/offline controls for the exact deployed version.
Ollama exposes API routes that some coding clients can use, but endpoint/schema compatibility does not establish a working client workflow. Choose the exact client version and follow its provider configuration, authentication and context guidance. For Claude Code, use the official Ollama integration recipe, including its documented native/compatible protocol requirements.
The local endpoint is http://127.0.0.1:11434; an OpenAI-compatible client normally
uses its /v1 route. Confirm the model ID with the running server. Then verify one
useful coding task, any required tool execution, error handling and restart behavior.
This guide's direct llama-bench and API smokes do not qualify every Cursor,
Continue or Claude Code version, nor measure coding-quality parity with hosted models.
Cursor is not an offline-local guarantee: its BYOK documentation (checked September 19, 2026) says requests pass through Cursor's servers for prompt building and Tab uses Cursor's own models. A localhost URL or your own API key does not establish that those features stay on-device. Verify the specific client/function/network path; do not expose an unauthenticated local model server to make a cloud-mediated client reach it.
Pinned, scoped client result: Open WebUI 0.10.2 passed discovery, visible
Qwen3.6 output and restart checks against the existing Ollama 0.32.15 route.
See the exact qualification and network boundary.
Docker must already be installed. A bridge container's host-gateway alias
cannot by itself reach a loopback-only Ollama listener. The measured host had
an existing LAN-reachable Ollama listener: it was not local-only. Do not
broaden an unauthenticated listener merely to copy this example.
docker run -d -p 127.0.0.1:3000:8080 \
--add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data \
--name open-webui \
ghcr.io/open-webui/open-webui@sha256:a26effeb220e132482bf7e0560b3404843e7bc40d23051144e062960df8df6b0Open http://localhost:3000. Privacy depends on the selected models, embeddings, tools and integrations. Provision artifacts first and verify that every enabled component stays local before claiming offline operation. The acceptance result covers the pinned local fixture, not every default setting, plugin or embedding provider.
For querying your own documents locally:
# 1. Pull an embedding model
ollama pull nomic-embed-text
# 2. Use Open WebUI's built-in RAG (easiest)
# or set up LangChain + ChromaDB for custom pipelinesFor a more current local embedding route, NVIDIA Llama Nemotron Embed 1B v2 now has a first-party CPU sanity pass. It ranked a relevant Strix Halo UMA passage above an unrelated passage, returned 2048-dimensional embeddings, and reproduced the exact vector in a fresh offline process. Follow the pinned reproduction and raw evidence. Keep this scoped to local functionality: a real Dutch/English corpus, long documents, batch throughput, memory, and ROCm acceleration still need measurement.
The measured Unsloth Strix Halo path uses an isolated ROCm 7.2 container. It completed a Radeon GPU gate, one-step SFT smoke, checkpoint inference, Q4_K_M export, ROCm llama.cpp inference, and a post-restart load from the host-persisted artifact.
Start there before attempting a longer QLoRA campaign. The smoke proves the workflow, not useful adaptation quality or large-model training performance.
The official Qwen3-ASR 0.6B Q8_0 GGUF now has a first-party llama.cpp b10107 Vulkan/RADV pass. It transcribed a known short English sample as Front, center. and returned the exact transcript after a fresh process. Use the exact command in the reproduction and raw evidence.
This proves a local load/transcription path, not general ASR quality. Dutch/multilingual accuracy, long audio, streaming, real-time factor, the 1.7B tradeoff, and word-error rate remain open; llama.cpp currently labels audio support experimental.
The official LFM2.5-VL 1.6B Q4_0 GGUF plus Q8_0 projector now has a first-party llama.cpp b10107 Vulkan/RADV pass. It correctly read the guide title, AMD Strix Halo platform, 101.0 t/s, 140.4 t/s, and 128 GB from the repository image, then repeated the exact answer in a fresh process. Use the exact command in the reproduction and raw evidence.
This is the low-download, lightweight vision route in the measured profiles. It is one image-functionality check, not a scored screenshot/photo/document benchmark.
kyuz0's ComfyUI toolboxes provide ROCm containers for Flux, Wan 2.2, and Hunyuan on gfx1151. For Vulkan-only: stable-diffusion.cpp works with the RADV driver.
AMD also publishes an official native-Windows ComfyUI route for Windows 11 24H2, current Adrenalin drivers, and ROCm 7.2.1, covering SDXL, Flux, and WAN workflows. That is useful official setup guidance, but this guide has not reproduced it on the Beelink or compared it with the measured Linux paths.
The current model evidence records a narrow Qwen3-TTS English speech generation and ASR back-check. A complete listen-answer-speak workflow, Dutch quality and streaming remain unqualified. Chatterbox and third-party voice applications are not established by that test.
The MAX+ 395 systems compared here share that APU; other Strix Halo variants must be checked separately. Memory options include 64GB, 96GB, or 128GB LPDDR5X-8000 depending on vendor and variant. The differentiators are memory size, form factor, cooling, ports, support, stock status, price, and how much public evidence exists for the exact chassis.
New storefront snapshot: September 19, 2026. Use the exact-SKU snapshot for source URLs, variant identifiers, stock/ETA, seller terms and unresolved checkout fields. US-facing/USD examples: GMKtec EVO-X2 128GB/2TB $3,649.99 (64GB/1TB is a different $2,199.99 offer), Beelink GTR9 Pro 128GB/2TB $4,349 pre-sale, Bosgame M5 128GB/2TB $2,999, Minisforum MS-S1 MAX 128GB/2TB $3,799 with early-October shipping, and Nimo 128GB/2TB $3,899.99 with delivery ETA unresolved. These are not delivered checkout quotes.
Framework's 128GB mainboard is not a complete PC; its current quote was unresolved. Corsair's exact 128GB/4TB SKU was out of stock with price unresolved. HP's selected laptop quote was unresolved. No older price was silently refreshed. GMKtec EVO-X3 uses MAX+ 395 in the observed offers; EVO-X2 evidence does not automatically qualify EVO-X3.
The July CSV and September 13 snapshot remain historical. Earlier unsupported successor/exclusivity claims stay withdrawn.
| System | Evidence depth and buying limitation |
|---|---|
| Beelink GTR9 Pro | Deepest first-party evidence here; not proof of superior hardware, support or value. Confirm board/NIC revision. |
| GMKtec EVO-X2 | Native/WSL2 community and external evidence; no first-party exact-retail-SKU buyer-path test. Keep 96GB external results separate from current 128GB offers. |
| Corsair AI Workstation 300 | Three community fleet systems plus power/RPC evidence; not three new matched retail setup campaigns. |
| Nimo AI Mini PC | Attributed large-model/server/MTP and thermal context; not a general support-quality claim. |
| Minisforum MS-S1 MAX | Windows LM Studio serving evidence; no matched native-Linux OEM speed ranking. |
| Bosgame / Framework / HP | Potential workload fits, but no same-shape first-party buyer-path reproduction here; mainboard and laptop categories need separate total-cost comparisons. |
Board/NIC revision note (Beelink GTR9 Pro): Some, but not all, original v1.0 systems with Intel E610-XT2 networking have reported NIC recovery/disconnection failures. Beelink has published update guidance and later introduced the v2.2 board with Realtek RTL8127 networking. Because field reports vary by unit and revision, confirm the exact board/NIC version with the seller and contact Beelink support with the serial number when troubleshooting. See the Beelink forum thread and Beelink's Q1 2026 BIOS summary.
How to compare the systems: compare memory configuration, dated delivered price/availability, evidence depth, cooling/thermals, firmware/support, ports, expandability and workload fit. Beelink's deeper first-party evidence is not a universal value win; Framework's modularity is not measured support superiority; network ports alone do not qualify a cluster. Affiliate commission is not a selection criterion.
Important: many Chinese mini PCs in this class, including Bosgame, GMKtec, and Beelink, appear to use closely related Sixunited platform designs. Do not assume every config is literally identical, but the first native GMKtec EVO-X2 community run reproduced the guide's Qwen3.6 Vulkan/RADV row within about 2%. Pick based on price, memory size, ports, cooling, and support.
| Route | Evidence here | Decision limit |
|---|---|---|
| Native Linux | Strongest first-party Vulkan/RADV evidence; scoped HIP and experimental vLLM routes | Follow the exact measured build/driver recipe; no universal kernel/HSA requirement |
| Native Windows | Community LM Studio serving/API report; AMD's external Windows 11 24H2 / ROCm 7.2.1 ComfyUI recipe | Not a local reproduction of image generation or matched Linux throughput test |
| WSL2 | Community GMKtec HIP baseline with a different TG512 workload | Separate from native Windows; not a general OS speed ranking or qualified vLLM path |
Model fit and setup effort depend on configuration and workload on all three routes; neither a fixed usable-memory allowance nor lower setup effort is established here.
Linux is strongly recommended for Strix Halo LLM work because it is the path with the strongest native Vulkan/RADV evidence. Windows is now represented by a community MS-S1-Max LM Studio report: Qwen3.6 Q4_K_M through LM Studio measured a 89.49 tok/s script average across mixed prompts, with long 512-token prompt rows around 69-70 tok/s. That is useful Windows serving/API evidence, not a same-shape comparison against native Linux
llama-bench. One GMKtec EVO-X2 community report also measured WSL2/HIP at 44.05 t/s on a TG512 Qwen3.6 generation-only run, while the same contributor's native Ubuntu Vulkan/RADV run measured 61.52 t/s on the guide's TG128 shape. Treat both as useful Windows-path baselines, not a clean same-machine Windows-vs-Linux conclusion.
New to local LLMs? Here's what the technical terms mean.
Click to expand glossary
APU -- Accelerated Processing Unit: a processor combining CPU and GPU capabilities. Strix Halo shares system memory between them; capacity depends on the selected 64GB, 96GB or 128GB configuration.
GGUF -- The GGML model-file format used by llama.cpp to store model weights and metadata needed to run inference.
Quantization -- Reducing the precision of model weights to use less memory and run faster. Common types:
- Q4_K_M -- 4-bit quantization, medium quality. Good balance of size and quality.
- Q8_0 -- 8-bit quantization. Better quality, ~2x the size of Q4.
- UD-Q4_K_XL -- Unsloth Dynamic 4-bit. Uses higher precision for important layers.
- BF16 -- A 16-bit floating-point format, reduced precision relative to FP32. Larger than the listed low-bit weight formats; task quality still requires evaluation.
MoE (Mixture of Experts) -- A model architecture where only a subset of parameters are active for each token. A "30B-A3B" model has 30 billion total parameters but only activates 3 billion per token, making it much faster than a dense 30B model while retaining most of the intelligence.
Dense Model -- A model where all parameters are used for every token. Slower but potentially smarter per parameter count. A dense 7B model uses all 7 billion parameters for every token.
Token -- The basic unit of text for LLMs. Roughly 3/4 of a word in English. "Hello, how are you?" is about 6 tokens.
Prompt Processing (pp) -- How fast the model reads your input. Measured in tokens/second. Higher is better. A pp of 800 t/s means the model can read ~600 words per second.
Token Generation (tg) -- Output tokens per second during generation. Responsiveness also depends on model loading, prompt processing, first-token latency and task length; a decode rate alone does not establish instant interaction.
Unified Memory -- System memory shared between CPU and GPU rather than separate discrete-GPU VRAM. Available model capacity depends on installed RAM, firmware and driver limits, weights, KV cache, runtime buffers and OS headroom; it is not a guaranteed 120GB model budget.
GTT (Graphics Translation Table) -- GPU mappings into system memory. The guide's large GTT/TTM limits belong to its recorded 128GB Beelink profile, not a universal setting or guaranteed usable model budget; preserve OS headroom and qualify other RAM sizes separately.
Vulkan -- A graphics/compute API. On Strix Halo, Vulkan is the most reliable backend for LLM inference via Ollama.
ROCm -- AMD's GPU compute platform (like NVIDIA's CUDA). Provides the HIP
backend for llama.cpp. Current native-gfx1151 builds should start without a
global HSA_OVERRIDE_GFX_VERSION; the older 11.5.1 setting belongs only to
the dated b8460/kernel 6.19.4 evidence. Vulkan RADV is still faster for the
measured generation rows, while HIP can win prompt-processing-heavy rows.
RADV -- Mesa's open-source Vulkan driver for AMD GPUs. AMD's only supported open-source Vulkan driver since AMDVLK was discontinued. Fastest measured default path here for Ollama, llama.cpp generation, and low-concurrency local API work.
AMDVLK -- AMD's former open-source Vulkan driver. Discontinued (last release April 2025). Uninstall it -- even inactive, its ICD file silently hijacks Vulkan and halves pp speed.
Ollama -- A tool that makes running LLMs as easy as ollama run model-name. Handles model downloading, GPU acceleration, and provides an API. Uses Vulkan on Strix Halo.
llama.cpp -- The open-source C++ library that powers most local LLM inference. Supports Vulkan, ROCm/HIP, and CPU backends.
Flash Attention -- An optimized attention algorithm that reduces memory usage and improves speed. Always enable it on Strix Halo (-fa 1 or OLLAMA_FLASH_ATTENTION=1).
tuned -- A Linux daemon that applies system performance profiles. accelerator-performance is used in some historical runs; its effect is workload-specific and the guide does not establish a universal tuned-only percentage gain.
What is the difference between Ollama and llama.cpp? Why is llama.cpp faster?
Ollama and llama.cpp are separate applications. Ollama adds model management, service APIs and backend selection; these measured GGUF routes use its bundled llama.cpp-derived runner. Do not assume every Ollama model uses an identical engine or commit to a standalone build.
So why can llama.cpp direct be faster on Qwen3.6 and Qwen3-Coder? Two reasons:
-
Wrapper/API overhead and run conditions. Ollama adds layers between you and the GPU: model loading, API translation, memory management, and service behavior. The normal Ollama 0.31.2 service path measured 60.57 t/s and passed iGPU, vision, restart, and reboot checks. A separate 0.31.1 run reached 71.82 t/s, while a later same-port/same-cache comparison put 0.31.1, 0.31.2, and 0.32.0 in the same 72.55-73.20 t/s class. Both service paths required
OLLAMA_IGPU_ENABLE=1so the Strix Halo iGPU was not dropped during GPU discovery. -
Bundled version. Ollama ships with a specific llama.cpp version baked in. Direct source builds can pick up new
llama.cppoptimizations earlier. The March b8298-to-b8460 jump gave +25% on some MoE Vulkan rows; later rows are tracked separately inBENCHMARKS.md.
Ollama prioritizes model/service management; standalone llama.cpp exposes explicit build and runtime controls. Neither a version number nor a newer source checkout guarantees a faster or more reliable workflow.
What should you use?
| Use case | Recommendation |
|---|---|
| Start from the measured beginner baseline | Ollama 0.31.2 system service -- the recorded path reached 60.57 t/s on Qwen3.6 with OLLAMA_IGPU_ENABLE=1 and survived restart/reboot. This does not qualify the revised install script or every client. Controlled isolated 0.31.1/0.31.2/0.32.0 binaries later measured in the same 72.55-73.20 t/s class. |
| Want explicit local API/runtime control | llama-server with Vulkan/RADV -- use the server measurements for the exact model, cache, API and concurrency profile. Direct llama-bench results are not server/API throughput expectations |
| Using kyuz0 containers | kyuz0 -- record the image digest and bundled runtime. A rebuilt moving tag is a new candidate, not an inherited qualification |
| Benchmarking | llama-bench -- direct runtime benchmark without HTTP/client serving; not an overhead-free or pure-GPU measurement |
Pinned llama-server route (scoped Qwen3-Coder acceptance):
# Start llama-server with your model (OpenAI-compatible API on port 8080)
# From its parent directory, enter the v0.4.1 Vulkan checkout built above.
# Candidate only, not an arbitrary latest binary.
cd llama.cpp-v0.4.1-vulkan
./build/bin/llama-server --help
AMD_VULKAN_ICD=RADV ./build/bin/llama-server \
-m ~/models/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf \
-ngl 999 -fa on --load-mode auto -c 8192 \
--host 127.0.0.1 --port 8080For a client that supports this endpoint, configure http://localhost:8080/v1
and verify its model ID, protocol and tool behavior. This pinned candidate has
passed the scoped direct/server acceptance controls in the September 19 qualification; it is not a speed
promise or a replacement for the reboot-qualified Ollama route.
This example is local-only. Remote serving needs a separately reviewed bind address, authentication, TLS and network access policy; consult the llama-server documentation before exposing it.
Can I run useful coding and chat models locally?
Yes. Qwen3.6-35B-A3B and Qwen3-Coder 30B-A3B are fast enough here for practical local chat, coding, scripts, and tool use. This guide measures local performance, not model quality against hosted systems.
Do I need Linux? Can I use Windows?
Linux gives the best-tested performance and the strongest native Vulkan/RADV evidence. Windows works for Vulkan-based inference via Ollama/LM Studio, and AMD's Adrenalin 25.8.1+ drivers added Variable Graphics Memory support for up to 96GB VGM. The guide now includes a Windows MS-S1-Max LM Studio serving/API report and a GMKtec EVO-X2 WSL2/HIP baseline. Treat both as useful Windows-path evidence, not proof that Windows matches native Linux llama-bench.
Is 128GB enough for the biggest models?
A 128GB configuration supports the specific large artifacts documented here, not every 70B Q4 or 120B model. Fit depends on exact weights, context/KV cache, runtime buffers, concurrent sequences, firmware and OS headroom. For larger models, two 128GB systems can provide 256GB aggregate physical RAM through a separately configured distributed runtime, subject to per-node limits and overhead. AMD demonstrated a 4-node cluster running a 1 trillion parameter model.
How does this compare to a Mac Studio?
Prices, availability, and external benchmark numbers change quickly; treat this as a dated comparison snapshot. Earlier May 2026 Mac Studio M4 Max 128GB price snapshots around $3,699 were useful for comparison, but high-memory Mac Studio availability changed quickly during the same month. Beelink's official GTR9 Pro US price snapshot is $4,349 (July 27, 2026), and this guide measures 71.82-101.0 t/s on the larger current Vulkan/Ollama headline paths, depending on model, backend, and quant, with ~215 GB/s bandwidth; Qwen3.6 also has an 81.30 t/s speed-first quant row, and smaller active-parameter MoE scouts can be higher. This guide does not establish a matched Apple-versus-Strix per-model winner. Strix Halo's advantages are Linux flexibility, ROCm/vLLM ecosystem access, dual 10GbE on some systems, and broader vendor choice with lower-priced alternatives.
Why is my speed lower than the guide says?
Common causes:
- Different power policy -- Record the active manager and compare it with the exact reference run. Only a tuned-profile reproduction requires
accelerator-performancewith power-profiles-daemon inactive. Another recorded policy is valid; no fixed percentage gain is promised. - Old Mesa drivers -- Check
vulkaninfo --summary | grep driverInfo. Should be Mesa 26.0.2+ from the kisak-mesa PPA; exact driver metadata is recorded per run when available. - Using Ollama instead of llama-bench -- Ollama and direct
llama-benchare different claim categories. The fully qualified 0.31.2 system service measured 60.57 t/s on Qwen3.6; controlled isolated 0.31.1/0.31.2/0.32.0 binaries later measured 72.55-73.20 t/s. All usedOLLAMA_IGPU_ENABLE=1. The 96-101 t/s Qwen rows are directllama-bench, not Ollama. - Unexpected GPU clock under load -- Inspect the correct GPU's
pp_dpm_sclkand compare with the reference workload, power limit and temperatures. The recorded Beelink 2900MHz state is not a universal idle or other-OEM requirement. - Wrong BIOS VRAM setting -- Check
free -h. On a 128GB system it should show roughly ~124-126GiB OS-visible memory; a 96GB system will be lower. If a 128GB box only shows ~31GiB, lower the UMA Frame Buffer reserve in BIOS. Use 512MB if available; if your vendor minimum is 2GB, leave it at 2GB. - Different model/quantization -- The 100.99 t/s Qwen3-Coder result is specifically Qwen3-Coder-30B-A3B Q4_K_S via RADV on official b9851 Vulkan. The older strict-clean b9179 row for the same speed-first quant remains 98.51 t/s. The 100.04 t/s result is a separate Qwen3-30B-A3B-Instruct-2507 IQ4_XS route. The balanced Qwen3-Coder UD-Q4_K_XL row is 96-99.6 t/s depending on build/repeat length. Larger or denser models are slower.
Can I use this for AI coding assistants like Cursor or Continue.dev?
Some client functions can use Ollama's compatible API, but this does not qualify every client, autocomplete feature or offline workflow. Start with the client-specific caveats, then test a named version, real task and tool round-trip. Cursor's documented server-mediated BYOK path must not be described as an all-local coding route.
Can I run image generation (Stable Diffusion, Flux)?
Yes. kyuz0's ComfyUI toolboxes provide ROCm containers for image and video generation on gfx1151, supporting Flux, Wan 2.2, and Hunyuan models.
Can I fine-tune models on this hardware?
Yes, with limitations. The guide now has a measured Unsloth/ROCm train-to-GGUF path: GPU detection, a one-step SFT smoke, checkpoint inference, GGUF export, ROCm llama.cpp inference, and post-restart artifact loading all passed on the retail Beelink. That proves the toolchain, not useful training quality or large-model speed. For longer campaigns, kyuz0's fine-tuning toolbox remains another ecosystem route. Full fine-tuning of large models is not practical compared with datacenter GPUs; use LoRA/QLoRA and validate held-out quality.
- kyuz0 -- Maintainer of the Strix Halo toolbox ecosystem, community standard containers
- lhl -- Deep performance research, rocWMMA patches, IOMMU/bandwidth testing
- pablo-ross -- Original GMKtec EVO-X2 setup guide
- TechnigmaAI / Hardware Corner -- Alternative optimization guide
- AMD -- Trillion-parameter LLM clustering article
- Lychee-Technology -- Pre-built llama.cpp binaries for gfx1151
- kisak-mesa PPA -- Latest Mesa drivers for Ubuntu
- GPUOpen-Drivers/AMDVLK -- Discontinued AMD Vulkan driver; kept here only as context for the ICD hijacking issue
Found something that's wrong, outdated, or missing?
See CONTRIBUTING.md for the full contribution path and the highest-value data currently wanted.
- Open a benchmark report with hardware, BIOS, kernel, driver, model, backend, command, and raw output.
- Open a power / efficiency report with wall-power or board-power readings, idle baseline, sample interval, raw readings, and the exact benchmark command.
- Use Discussions for setup questions, early results, and comparisons.
- PRs welcome -- especially from other Strix Halo systems (Framework, GMKtec, HP ZBook).
- If you find a new optimization, include before/after benchmarks.
This guide stays free and evidence-first. If it saved you setup time or helped you choose hardware/software, the most useful support is a GitHub star, a benchmark report, a correction, or a pull request.
Financial support may fund hardware, storage, model downloads, testing time, and ongoing maintenance, but it does not influence benchmark conclusions. See SUPPORT.md for the funding policy.
For private setup help, an independent reproduction, or a scoped OEM
buyer-path pilot, see SERVICES.md. The public guide remains
free, and paid work does not buy positive conclusions.
- Current Qwen3.8 decision layer published: the guide now separates its measured official Ollama route from external 262K-class validation, stock community MTP, tuned ROCm/DFlash reports, and an unpublished-sidecar performance lead. A matched route ladder is the next performance test; community values are not promoted into guide-owned headlines.
- Selected public-state checks: the validator checks evidence-review age, required text fragments on named local pages, coverage arithmetic and declared affiliate-registry state. It does not prove all numbers or prose are synchronized, absence of undisclosed links, rendered website correctness or deployed-site freshness. Scheduled validation is not a substitute for claim/source review.
- Buyer, contributor, and vendor routes tightened: the protected SEO/hook introduction now leads directly into audience-specific decisions; a dedicated Qwen web page and share card, current responsible-share copy, current traction evidence, contribution guardrails, and explicit affiliate-ranking firewall remove ambiguity without turning the README into an advertisement.
- mottledMantis' Gemma row fully imported: preserved the issue CSV and provenance, added the stock b9235 GMKtec result (1209.08 pp512 / 53.02 tg128) to community data, benchmarks, current-model guidance, contributor credit, and the system matrix. Decode is in the same practical band as the newer 54.18-55.45 t/s Beelink controls; unmatched model bytes/hash, build, Mesa, mmap, repeats, and host state prevent an OEM ranking.
- Qwen3.8 evidence split sharpened: the measured Ollama 0.32.13 Beelink route remains the buyer baseline through 50,059 prompt tokens. A corrected 96GB GMKtec public package adds 13/13 exact retrieval up to a 261,130-token needle and a 6/6 image pilot, proving 262K-class operation can work while remaining an advanced patched-HIP route.
- HIP correctness warning corrected: the earlier b10046 result is now scoped to small-model full-UMA allocation evidence. Open
llama.cppissue #26209 and PR #25863 make a stock b10622-versus-fix exact-output A/B the highest-priority runtime test before long-context, vision, or multi-slot HIP guidance is promoted. - Runtime targets refreshed:
llama.cppv0.3.0 / b10622 and Ollama 0.32.15 are current checked releases, not locally qualified replacements. The Ollama qualification now includes the reported Qwen3.8 Anthropic-compatiblexhighfailure mode. - Community map expanded to 13 systems/sources and 10 contributors: the Kyanite Labs GMKtec evidence package is counted separately from first-party and other GMKtec owner rows.
- Community map expanded to 12 systems/sources and 9 contributors: Ornery_Specialist_83 independently reproduced the first-party Beelink Qwen3.6 ~62-63 t/s figure on a second GTR9 Pro unit and documented the silent
llama.cpp-ub > -bclamp (+28.9% pp512 after correction) plus a measured stock-Mesa-to-kisak uplift (+16.9% pp512). Added to the rules of thumb, troubleshooting, community results, and the evidence matrix.
- Official storefronts rechecked: the buyer table now records the displayed configuration, price, and availability state for GMKtec, Bosgame, Framework, Beelink, Corsair, Minisforum, Nimo, and HP. GMKtec's official EU listing is recorded separately because its 128GB price and availability differ materially from the US listing. The configuration-level source, price, and fulfillment state are preserved in
data/buyer_price_snapshot_2026-07-27.csv. The dated snapshot is deliberately not a tax-, shipping-, coupon-, or regional-cart quote. - Material changes reflected: Beelink's listed 128GB+2TB price is now $4,349 and remains pre-sale with a stated 35-day ship window; Corsair's listed 128GB/4TB price is $3,399.99 and currently out of stock; and Minisforum's listed 128GB+2TB Max AI Compute Edition price is $3,639 with mid-August estimated shipping. The current Beelink value is also used in the hardware comparison and break-even examples.
- Ollama 0.32.3 isolated qualification passed: same-cache Qwen3.6 measured 73.13 t/s versus 73.20 t/s on 0.31.2 with exact outputs. Qwen2.5-VL fully offloaded to the iGPU, read the test image correctly, and repeated after process restart. The installed 0.31.2 service remains the default until current 0.32.5 passes a normal package upgrade and full-host reboot.
- Official
llama.cppb10107 capability routes added: LFM2.5-VL 1.6B correctly read the guide image and Qwen3-ASR 0.6B transcribed a short known English sample. Both reproduced exactly after a fresh process. These are scoped function/restart smokes, not broad vision or speech-quality benchmarks. - Modern local embedding route added: Llama Nemotron Embed 1B v2 ranked a relevant Strix Halo UMA passage above an unrelated passage, returned 2048-dimensional embeddings, and reproduced the exact vector offline. The measured route used CPU and remains a tiny retrieval sanity check rather than multilingual-corpus or ROCm evidence.
- PR #25666 independently checked: the Qwen3.6 MTP stock-versus-PR A/B preserved exact generated outputs and 80.187% draft acceptance. Warm repeats were only 0.55% faster on the PR head, so the result is recorded as correctness/no-regression evidence rather than a performance headline.
- Strict three-system campaign imported: Fail-Safe's matched Corsair AI Workstation 300 sweep now has normalized CSV rows, two generated charts, a complete raw bundle, analyzer, contamination checks, cap/reset harness, and bounded stock controls.
- Scoped buyer guidance: 2400 MHz was the best measured conservative tradeoff on this fleet; moving to 2600 MHz added 6.39% prompt throughput and 0.87% generation while mean AMDGPU socket power increased 21.70%. This is not a universal cap recommendation.
- Root-cause framing corrected: historical logs showed missing
ec_su_axb35modules and failed dependent services after kernel updates on two systems. The guide records this as a plausible major confounder, keeps the root cause unresolved, and tracks the then-open fan-reset patch (merged August 20; installed-version qualification remains separate).
- Current direct capacity: DeepSeek V4 Flash 284B
UD-IQ2_XXSloaded and generated from a pinned 90.86GB ordinary GGUF on officialllama.cppb10034 at 155.64 pp512 / 13.27 tg128. A deterministic smoke answered correctly; this remains low-bit capacity/current-model evidence rather than a speed or broad quality recommendation. - Current agent/server route: Step 3.7 Flash 198B-A11B ROCmFPX Q3 plus its Q8 MTP draft measured 34.50 t/s at 4K and 33.83 t/s at 16K, passed a native tool call, and allocated 256K context on one 128GB Beelink.
- ROCm 7.14 workaround reproduced: in the pinned official ROCm 7.14 / PyTorch 2.11 / vLLM image, enabling hipBLASLt improved Qwen3-0.6B FP16 aggregate throughput by 40.50% / 38.96% / 41.54% at concurrency 8/9/16. Concurrency 1 was unchanged and 4 was slightly slower, so this is a batch-8+ server profile rather than a universal tuning rule.
- HIP integrated-host-buffer allocation reproduced (historical scope): official
llama.cppb10046 detected the full Strix Halo UMA pool and allocatedROCm_Hostmodel/output/compute buffers without a gfx-version override. The 2026-08-25 recheck limits this to small-model allocation/setup evidence because later upstream long-context/multimodal correctness reports were outside this smoke's coverage. - Current runtime checks: official b10034 still showed the Vulkan MoE 8-to-9 sequence cliff on two model shapes; a same-cache Ollama 0.31.1/0.31.2/0.32.0 comparison found no material version regression; current Nemotron Omni, Cascade, AgentWorld, Audex, and CHADROCK routes were also rechecked with scoped evidence.
- Official b9888 Vulkan sentinel checked: Qwen3-Coder 30B-A3B
Q4_K_Smeasured 1404.73 pp512 / 98.12 tg128 r50 on b9888 with the b9851/b9859-matching command shape, and 98.59 t/s generation-only.UD-Q4_K_XLmeasured 1410.82 pp512 / 96.53 tg128 r5. - Claim boundary preserved: b9888 is the latest official
llama.cppruntime control, but it does not replace the stronger b9851 Qwen3-Coder 100.99 t/s speed-first headline. - DeepSeek V4 Flash route triaged again: current routes include very large ordinary GGUF artifacts, a 92.8GiB
IQ2_MGGUF candidate whose download was too slow to complete in this pass, and a smaller 46.98GiB REAP route that needs a separate ds4 runtime. No local pass/fail or speed claim is published yet.
- Ollama easy-path sanity check updated: a user-local Ollama 0.31.1 binary on port 11435 measured 71.82 t/s warm API generation mean on
qwen3.6:35b-a3b, with a 71.62-72.05 t/s warm range after one cold run. This is Ollama API / buyer-path evidence, not directllama-bench. - Important setup fix: current Ollama builds can drop the Strix Halo iGPU during GPU discovery unless
OLLAMA_IGPU_ENABLE=1is set. The setup script and README environment guidance now include that variable. - Official b9859 Vulkan sentinel checked: Qwen3-Coder 30B-A3B
Q4_K_Smeasured 1413.38 pp512 / 98.48 tg128 r50 on b9859 with the b9851-matching command shape, and 99.09 t/s generation-only.UD-Q4_K_XLmeasured 1411.76 pp512 / 97.01 tg128 r5; Gemma 4 26B-A4BUD-Q4_K_Mmeasured 1323.39 pp512 / 54.18 tg128 r5. - Claim boundary preserved: b9859 is a useful current-runtime control, but it does not replace the stronger b9851 Qwen3-Coder 100.99 t/s speed-first headline.
- Fail-Safe added a Corsair MiMo V2.5 capacity row: issue #26 reports MiMo-V2.5
UD-IQ2_M, described byllama-benchasmimo2 310B.A15B IQ2_M - 2.7 bpw, on Corsair AI Workstation 300ai-2with Fedora 44, kernel 7.0.12, Mesa RADV 25.3.6 inside the kyuz0 Vulkan container, and IOMMU off. - Claim boundary preserved: the pasted CSV row is 30.65 pp512 with
n_gen=0, so this guide treats it as prompt-processing capacity evidence, not as a tg128/generation headline. - Vendor/buyer value: the row includes wall-power/GPU telemetry: Home Assistant wall power 29.7-162.4 W with a 114.14 W mean, GPU edge 40-75 C, and GPU socket power 14.098-104.021 W with a 70.92 W mean. This strengthens the Corsair evidence layer beyond Qwen speed rows.
-
Qwen3-Coder direct speed-first row crossed 100 t/s: the exact
Q4_K_SQwen3-Coder 30B-A3B file measured 1423.05 pp512 / 100.99 tg128 r50 on the officialllama.cppb9851 Ubuntu Vulkan release binary with explicit-dev Vulkan0. This is directllama-benchevidence, not MTP/server speculation. The older b9179 strict-clean 98.51 t/s row remains preserved as historical context. -
Balanced Qwen3-Coder latest-build control improved:
UD-Q4_K_XLmeasured 1416.79 pp512 / 99.55 tg128 r5 on b9851. Treat this as a short latest-build control unless a longer repeat is needed. -
Gemma direct control stayed secondary: Gemma 4 26B-A4B
UD-Q4_K_Mmeasured 1326.52 pp512 / 55.45 tg128 r5 on b9851, so the existing Gemma 4 26B-A4B QAT MTP route remains the useful high-throughput Gemma path. -
Community map expanded to 11 systems/sources: papagenic contributed a Minix Elite ER939 Ai Ollama 0.30.10 report for
qwen3.6:35b-a3bon Ubuntu 26.04, kernel 7.0.0-22, Mesa 26.1.3, BIOS UMA 1G, and IOMMU disabled. -
Claim boundary preserved: the row is useful buyer-path evidence for Minix/Ollama/Ubuntu 26.04, but it is not a direct
llama-benchheadline because backend/Vulkan ICD, script details, repeats, and warm/cold state are not yet confirmed. -
Public traction snapshot refreshed: the guide reached 192 GitHub stars, 9 forks, and 4 watchers in the 2026-07-02 GitHub API snapshot. This remains a small-niche demand signal; the main value is still reproducible public evidence that reduces buyer setup friction.
-
Runtime watchlist refreshed: the
llama.cppb9851 sentinel is now measured; the later 2026-07-02 check separately measuredllama.cppb9859 and a user-local Ollama 0.31.1 sanity path.
- Public traction snapshot refreshed: the guide reached 160+ GitHub stars and 10 Strix Halo-class systems represented in the 2026-06-21 snapshot. This is a small-niche demand signal; the main vendor value remains reproducible public evidence that reduces buyer setup friction.
- ROCmFP4 / CHADROCK advanced lane added:
ROCMFP4_CHADROCK.mdtracks therocmfp4-llama/ tuned-GGUF route separately from the beginner Vulkan/RADV/Ollama path and from first-party directllama-benchheadline rows. - ROCmFP4 Crown Halo smoke added: the
jcbtc/qwen3.6-35b-a3b-crown-halo-mtp-dynamicartifact now has first-party Beelink load/API/MTP smoke evidence. It runs, but the high-speed community dynamic-MTP band is not yet reproduced on this HIP-only Beelink route. - CHADROCK ACE/SABER stability profile added: the
jcbtc/chadrock-35b-ace-saber-rocmfp4-mtpartifact loads and serves on the guide's Beelink/RADV path with the pinnedciru-ai/ROCmFPXrunner. The exact 3946-token reference profile averaged 141.37 t/s over three gen512 repeats at 100% draft acceptance; 1K/8K/16K profiles measured 78.00/83.85/107.23 t/s. This is strong advanced-lane evidence and practical acceptance guidance, but still separate from directllama-benchrows. - Nemotron 3 Nano Omni b9747 smoke added:
unsloth/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUFMXFP4_MOEloads and runs directly on the officialllama.cppb9747 Vulkan binary at 1277.60 pp512 / 56.56 tg128. This is current NVIDIA Omni/FP4 support evidence, not a new speed headline. - Current model watchlist refreshed: Kimi-K2.7-Code, GLM-5.2, MiniMax-M3, Nemotron 3 Nano Omni NVFP4/MXFP4, DeepSeek V4 Flash REAP,
llama.cppb9747, and Ollama 0.30.10 were triaged for practical guide value.
- GMKtec EVO-X2 NixOS evidence package linked: ciru-ai published a public artifact for GMKtec EVO-X2 / NixOS / IOMMU-on / NPU-aware Strix Halo benchmarking, including sanitized CSV/SQLite exports and a compact public metrics index.
- NPU sidecar evidence added: the artifact reports +3.29% main 64k iGPU workload latency with concurrent NPU load versus +68.96% with a comparable iGPU auxiliary load, plus FastFlowLM-NPU LFM2.5 1.2B at 32k around 1646 prompt tok/s and 38.18 decode tok/s.
- ROCmFP4 and quality-eval context added: selected Chadrock/Qwopus/Qwen3.6/Gemma/CrownV7 rows are linked as community tuned-route evidence with quality metrics. These are not first-party Beelink direct
llama-benchheadlines.
- Gemma 4 26B-A4B QAT MTP repeat documented: the matched-head
llama-serverroute measured 102.69 t/s cold, 107.42 t/s with only T3 left among known local services, and 110.00 t/s as the best repeat onac4cddeb0. This is server/speculative evidence, not a directllama-benchreplacement. - Current setup routing improved: the README now sends new users to
setup.shfirst, then Ollama/Open WebUI for local chat, and only then to directllama.cpp, MTP, Lemonade, or ROCm/vLLM paths when they need benchmark or server control. - Community Beelink CachyOS ROCm/ZenDNN row added: devoidfury contributed a second Beelink owner stack with CachyOS, kernel 7.0.11, ROCm 7.2.4, local ZenDNN, and Qwen3.6 27B MTP
UD-Q6_K_XL. ROCm roughly doubled pp5000 versus Vulkan on that setup while decode stayed around 8 t/s, making it backend-crossover evidence rather than a decode headline.
- Latest-control rows added: Qwen3-30B-A3B-Instruct-2507 stayed above 100 t/s at 100.38 t/s, LFM2.5 stayed in the 170 t/s class at 171.17 t/s, and Nemotron 3 Super stayed in the 18 t/s capacity class at 18.24 t/s.
- Gemma 4 QAT direct route added: Gemma 4 26B-A4B IT QAT
UD-Q4_K_XLmeasured 74.80 t/s direct and became the practical current Google-model row, replacing the older non-QAT Gemma 4 row as the recommendation. - Negative/control evidence preserved: Qwen3.6 27B NVFP4 loaded but was not a speed route on this Vulkan/RADV path, and the MTP smoke result remained too slow to recommend.
- No Vulkan/RADV sentinel regression found: Qwen3-30B-A3B-Instruct-2507 measured 103.18 t/s, the exact Qwen3-Coder Q4_K_S speed-first file reproduced the 98 t/s class at 98.02 t/s r50 and 98.49 t/s generation-only, and Qwen3-Coder UD-Q4_K_XL stayed in the 96-97 t/s class.
- Small-MoE and capacity rows held up: LFM2.5 measured 176.48 t/s, and Nemotron 3 Super measured 18.93 t/s on the b9544 control.
- New speed and capacity scouts added: LFM2.5 8B-A1B reached 170.02 t/s generation-only, Nemotron 3 Nano reached 75.97 t/s, and Nemotron 3 Super 120B-A12B reached 18.43 t/s direct on one 128GB Strix Halo system.
- Failure data kept: DeepSeek V4 Flash and other large-model routes stayed documented as blocked or impractical where model distribution, architecture support, or storage/runtime requirements prevented a clean local benchmark.
- Nimo AI Mini PC evidence added: boxwrench contributed a Ryzen AI MAX+ 395 / Radeon 8060S / 128GB Nimo bundle in issue #4 with system metadata, reproducibility notes, raw benchmark rows, thermal telemetry, and model-specific follow-ups.
- Community map now covers 8 systems: the guide now tracks Beelink first-party data plus three Corsair systems, two GMKtec sources, MS-S1-Max, and Nimo community evidence. This was later expanded to 11 systems/sources with a second Beelink owner stack, ciru-ai's GMKtec EVO-X2 NixOS/NPU artifact, and papagenic's Minix Elite ER939 Ai Ollama report.
- Large-model buyer context added: Nimo rows cover Qwen 3.5/3.6 35B, Qwen 122B-class serving, Qwen3-Coder-Next, StepFun Step-3.7-Flash, GPT-OSS/Gemma notes in the raw bundle, and DFlash negative/control evidence. These are community serving/eval rows, not first-party direct
llama-benchheadline claims. - Gemma 4 QAT follow-up added: boxwrench added Gemma 4 12B, 26B-A4B, and 31B QAT Q4_0 rows with matched MTP assistant-head comparisons. The useful lesson is not a homepage headline; it is that matched QAT assistant heads can materially improve single-stream decode and acceptance. Atomic PR #26 later fixed the reported
PARALLEL=2Gemma 4 MTP crash, so fresh post-merge 2-slot community numbers are now the useful follow-up. - Vendor/adoption value improved:
COMMUNITY_NIMO.mdsummarizes what the Nimo bundle proves and what it does not prove, so vendors/reviewers can see how additional hardware reduces setup and buyer uncertainty without turning community data into endorsement language.
- First local direct 100 t/s row added: Qwen3-30B-A3B-Instruct-2507
IQ4_XSreached 100.04 t/s tg128 r50 and 1416.03 t/s pp512 on llama.cpp b9467 / Vulkan/RADV. This is a directllama-benchresult, but it is explicitly kept separate from the Qwen3-Coder 98.51 t/s headline and the Qwen3.6 MTP server route. - Windows LM Studio evidence added: bennos1911 contributed a Minisforum MS-S1-Max Windows 11 / LM Studio 0.4.15 / Qwen3.6 Q4_K_M serving report with benchmark script, CSV output, and hardware telemetry. It is documented as Windows-path evidence, not as a same-shape Linux
llama-benchcomparison. - Tuned GMKtec 100 t/s report added: Look_Over_There contributed a Reddit GMKtec EVO-X2 Qwen3-Coder
Q4_K_Sb9467 report where most short-context runs were around 99.90 t/s and the best observed run reached 100.0 t/s after about 10 runs. It is explicitly labeled as tuned thermal/power-policy evidence because the system had heatsink repaste, memory-pad reseating, lower reported temperatures, and GPU/CPU high-performance policy. - Community signal updated: community results now include Beelink first-party data plus Corsair, GMKtec, and MS-S1-Max community evidence across Linux Vulkan/RADV, WSL2/HIP, Windows LM Studio, wall-power, RPC, USB4 tuning, and tuned thermal/power-policy reports.
- Upstream watch rechecked: ROCm production remains 7.2.4, vLLM has moved to 0.22.1, and the previous isolated Ollama 0.24.0 check did not change the then-installed Ollama 0.23.1 guidance at that time.
- No new headline from latest llama.cpp direct reruns: the 2026-06-01
de6f727aaQwen3-Coder direct check measured 95.55 t/s tg128 withmmap=0, so the direct headline stays at 98.51 t/s on the b9179 strict-clean speed-first row. - Qwen3.6 27B MTP control confirmed: the latest-build rerun measured 7.61 t/s without MTP and 14.69 t/s with MTP, so the official dense 27B Q8_0 route remains useful negative evidence rather than a speed candidate. Details live in
PERFORMANCE_NOTES.md. - Community hygiene improved: responsible-sharing guidance was added to
CONTRIBUTING.mdandSHARE.md, and Fail-Safe's.gitignorePR added macOS/Windows cache-file ignores for cleaner community contributions.
- MTP route crossed 100 t/s broad average: llama.cpp b9360 (
6b4e4bd58) with Mesa/RADV 26.1.1 pushed Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn to 101.16 t/s best six-prompt average withdraft-n=2,--poll 100, and-ub 1024. Three t16 repeats landed at 101.15 / 101.10 / 101.06 t/s. - At that date, still not a direct 100 t/s claim: b9360 direct Qwen3-Coder Q4_K_S measured 97.23 tg128, and UD-Q4_K_XL measured 92.60 tg128. The Qwen3-Coder direct row remained b9179 Q4_K_S 98.51 t/s and balanced b9049/b9010 96-97 t/s until the later 2026-06-02 guide update added a separate Qwen3-30B-A3B-Instruct-2507 direct 100.04 t/s row.
- Added raw evidence under
data/raw/2026-05-27/latest-llamacpp-b9360/and updatedBENCHMARKS.md,MTP_SPECULATIVE_DECODING.md,data/benchmarks.csv,data/mtp_speculative.csv, anddata/headline_claims.csv.
- MTP route improved again: llama.cpp b9334 (
192d8ae) with Mesa/RADV 26.1.1 raised the local Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn six-prompt average to 98.57 t/s withdraft-n=3; the best b9334 prompt reached 116.75 t/s. - At that point, still not a broad 100 t/s claim: repeated b9334 MTP runs landed around 97.76-98.57 t/s depending on thread/poll settings. This was close and valuable, but later got superseded by the b9360
-ub 1024MTP rerun. - Direct latest-stack check stayed below the headline: llama.cpp b9334 direct Qwen3-Coder Q4_K_S measured 96.27 tg128, and UD-Q4_K_XL measured 94.15 tg128. The existing direct headline remains the b9179 Q4_K_S 98.51 t/s strict-clean row and the balanced b9049/b9010 96-97 t/s UD row.
- Added raw evidence under
data/raw/2026-05-26/latest-llamacpp-b9334/and updatedBENCHMARKS.md,MTP_SPECULATIVE_DECODING.md,data/benchmarks.csv,data/mtp_speculative.csv, anddata/headline_claims.csv.
- GMKtec Qwen3-Coder full row: mottledMantis added the requested full
pp512/tg128Qwen3-Coder 30B-A3B UD-Q4_K_XL b9235 run on GMKtec EVO-X2: 1157.29 pp512 / 91.40 tg128. - Command-shape caveat preserved: the full row used
-b 512 -ub 512,flash_attn=0, anduse_mmap=1, so it is documented as portability and flag-sensitivity evidence, not as an apples-to-apples replacement for the Beelink headline row. - Added the raw CSV under
data/raw/2026-05-19/community-gmktec-qwen-coder-issue17/and updatedCOMMUNITY_RESULTS.md,CONTRIBUTORS.md,BENCHMARKS.md,SHARE.md, anddata/community_results.csv.
- GMKtec MTP reproduction: mottledMantis reproduced the exact Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn route from the guide on a GMKtec EVO-X2 with Ubuntu 26.04, kernel 7.0.0-15, Mesa RADV 26.0.3, and llama.cpp b9235.
draft-n=2averaged 93.29 t/s over six prompts, slightly above the local Beelink b9235 92.30 t/s row. This strengthens the MTP route but still is not a broad 100 t/s claim. - GMKtec Qwen3-Coder baseline: the same contributor added a Qwen3-Coder 30B-A3B UD-Q4_K_XL b9235 generation-only row at 92.11 t/s. This is lower than the Beelink/Corsair b9049 rows, but useful as a GMKtec/latest-stack baseline and as evidence that build/model/source/host-state details matter.
- Added raw community artifacts under
data/raw/2026-05-19/community-gmktec-*, updatedCOMMUNITY_RESULTS.md,data/community_results.csv, anddata/mtp_speculative.csv.
- Latest MTP master rerun at the time: llama.cpp b9235 (
d14ce3dab) raised the Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn six-prompt average to 92.30 t/s withdraft-n=3. The fastest prompt in this rerun was 109.21 t/s; the older b9187 sweep held the best single-prompt MTP result at 110.61 t/s until the later b9334 rerun. - Official 27B MTP Q8_0 checked:
ggml-org/Qwen3.6-27B-MTP-GGUFwas tested on the same b9235 Vulkan/RADV stack. It reached 7.74 t/s without MTP and 14.59 t/s with the best MTP setting, making it a useful negative result rather than a speed/headline candidate. - Added raw evidence under
data/raw/2026-05-19/and updated the MTP CSV/claim index.
- MTP speculative decoding documented: Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn reached 90.80 t/s average over six
llama-serverprompts and 110.61 t/s on the best prompt. This is a server/speculative result, not a replacement for the directllama-benchheadline and not a broad 100 t/s average. - Qwen3-Coder ngram server route checked: Qwen3-Coder Q4_K_S with
llama-serverngram speculation reached 95.21 t/s average, below the direct 98.51 t/s speed-first headline. - Community wall-power cross-section added: Fail-Safe's Corsair AI Workstation 300 power data now covers Qwen3-Coder, Qwen3.6, gpt-oss-120b, and Qwen3-Coder-Next, with sustained generation rows from about 137-174 W and 1.6-3.4 J/token.
- Contribution flow tightened:
CONTRIBUTING.mdand the benchmark/power issue templates now ask for the metadata needed to turn community reports into structured evidence. - Added raw evidence under
data/raw/2026-05-17/anddata/raw/2026-05-10/community-power-issue6/.
- Qwen3-Next 80B improved: llama.cpp b9172 with Vulkan/RADV confirmed 59.06 t/s tg128 and 751.70 pp512, replacing the old 54.92 t/s b8933 row as the historical May 80B Qwen-family result.
- Qwen3-Coder speed-first peak: b9179 plus Q4_K_S confirmed 98.51 t/s r50 after fixing the
tuned/power-profiles-daemonconflict and pausing benchmark noise. This is a speed-first quant row, not the balanced UD default. - No main headline speedup from b9172: Qwen3-Coder UD, Qwen3.6, and gpt-oss-120b did not beat the current b9049/b9010 balanced headline rows.
- Ollama 0.24.0 isolated check: Qwen3.6 measured 49.05 t/s warm generation, effectively identical to the same-prompt Ollama 0.23.1 control at 49.09 t/s.
- ROCm nuance confirmed again: Lemonade ROCm b1259 won Qwen3-Next pp512 (800.38 vs Vulkan 751.70), but Vulkan/RADV won tg128 (59.06 vs ROCm 49.57). The guide now avoids "RADV wins everything" wording and keeps the beginner rule focused on generation-heavy GGUF chat/coding.
- Added raw evidence under
data/raw/2026-05-16/.
- llama.cpp b9049 rerun: Qwen3-Coder 30B-A3B measured 96.76 t/s generation in the max-performance guide-flags confirmation; Qwen3.6 35B-A3B measured 62.56 t/s on the balanced UD row and 81.30 t/s on the speed-first Q4_0 row.
- Ollama upgraded to 0.23.1: Qwen3.6 API warm average remained 50.51 t/s, matching the 0.21.2 baseline.
- gpt-oss-120b local check: ggml-org MXFP4 split GGUF loaded locally and measured 55.57 t/s tg128, 726.99 t/s pp512, and prompt processing through 65K tokens via llama.cpp b9049 Vulkan/RADV.
- HIP/Vulkan workload split added: local spot check shows HIP winning pp16384 and Vulkan winning tg128 on both Qwen3.6 and Qwen3-Coder rows; see
BACKEND_CROSSOVER.md. - ROCm/vLLM bugwatch added: current upstream ROCm/vLLM release and issue status moved to
ROCM_VLLM_BUGWATCH.md. - Headline range tightened: current direct llama.cpp headline range became 63-97 t/s for the balanced rows. The previous b9010 Qwen3-Coder peak of 97.24 t/s remains in the data as historical evidence, and Qwen3.6 added an 81.30 t/s speed-first quant row.
- Added clean raw evidence under
data/raw/2026-05-07/latest-stack-rerun/clean-b9049-rerun/.
- Qwen3-Coder 30B-A3B benchmark updated: controlled b9010 Vulkan RADV rerun averaged 97.24 t/s generation and 1346 pp512 across two separate
-r 20runs. - Qwen3.6 UD rerun: controlled b9010 Vulkan RADV rerun averaged 63.06 t/s generation and 1109 pp512 across two separate
-r 20runs. The old "UD costs 13%" warning was not reproduced on the current stack. - Ollama Qwen3.6 rerun: controlled API test averaged 50.5 t/s warm generation across 10 runs, replacing the older 45-46 t/s easy-path claim. Current Qwen3.6 Ollama overhead is about 20-21%, not ~30%.
- Multi-user Qwen3.6 serving:
llama-servercontinuous batching reached 162 t/s aggregate at-np 8with ~0.31 s TTFT, then plateaued at 166 t/s at-np 16. - Multi-user Qwen3-Coder serving:
llama-servercontinuous batching reached 173 t/s aggregate at-np 8;-np 16regressed to 130 t/s aggregate. - Local long-context prompt scaling: Qwen3.6 processed 64K prompts at 740 t/s and Qwen3-Next 80B processed 64K prompts at 544 t/s on Vulkan RADV.
- Filled-KV decode: Qwen3.6 generated 41.4 t/s after a 64K f16 prompt; q4_0 KV raised decode to 51.3 t/s but increased total request time from 73.5 s to 90.0 s because prompt ingest slowed.
- 128K filled-KV decode: Qwen3.6 generated 32.2 t/s after 128K and Qwen3-Next 80B generated 29.1 t/s after 128K, both without truncation.
- Real-corpus 64K check: using this guide's own documentation as the prompt, Qwen3.6 decoded at 40.8 t/s after ~65K tokens and Qwen3-Next 80B at 37.8 t/s after ~64K tokens. Prompt ingest was slower than synthetic prompts, but decode-after-fill barely changed.
- ROCm HIP spot check: current local HIP b8460 path is usable with the HSA override, but remains behind Vulkan for short-context tg: Qwen3.6 52.7 t/s and Qwen3-Coder 73.7 t/s.
- At that point, the headline range moved from 65-87 t/s to 65-97 t/s. This was later tightened to 63-97 t/s for balanced rows in the 2026-05-07 rerun, extended to 63-98.5 t/s by the 2026-05-16 Qwen3-Coder speed-first Q4_K_S row, extended with a separate 100.04 t/s Qwen3-30B-A3B-Instruct-2507 IQ4_XS direct row on 2026-06-02, and later lifted to 101.0 t/s for Qwen3-Coder Q4_K_S on official b9851. The previous 87.11 t/s result remains in
data/benchmarks.csvas historical-local data. - Added raw benchmark output under
data/raw/2026-05-03/so the new headline can be audited.
- Beelink price audit: The official GTR9 Pro page now lists the 128GB+2TB variant at $4,399 with a $4,699 compare-at price. Earlier snapshots and checkout prices were substantially lower, so the guide no longer uses the old lower Beelink figure as a headline claim.
- Removed stale "Strix Halo is much cheaper than DGX Spark" wording. At the current Beelink official price, the gap to DGX Spark is only about $300, while lower-priced Strix Halo systems still exist.
- Updated cost, hardware comparison, buying guide, and Mac Studio FAQ language so price-sensitive claims are clearly date-bound.
- AMDVLK ICD hijacking discovered: All "pp regression" findings (b8460 vs b8933, Mesa 26.0.2 vs 26.0.5) were caused by AMDVLK's
/etc/vulkan/icd.d/amd_icd64.jsonsilently overriding RADV. No actual regression exists. Corrected on #22375. All benchmarks re-verified on actual RADV - Qwen3.6-35B-A3B benchmark: 64 t/s tg, 1064 pp512 via Vulkan RADV. Drop-in replacement for Qwen3.5 with better coding/reasoning quality, identical speed. The old UD-Q4_K_M penalty note is superseded by the May 2026 controlled rerun.
- Qwen3-Next 80B-A3B benchmark: 55 t/s tg, 657 pp512 via Vulkan RADV (b8933). 80B MoE (3B active) with 256K context window. The separate Qwen3-Coder-Next 80B-A3B MoE row measured 38 t/s (architecture label corrected September 5)
- Gemma 4 26B-A4B benchmark: 48.5 t/s tg, 1142 pp512 via Vulkan RADV (b8933). Included a KV-cache warning; September 2026 correction: the external ~3.5× KL ratio compares Gemma 26B with Gemma 31B, not Qwen or downstream task-quality loss.
- Llama 4 Scout 109B benchmark: 18.3 t/s tg, 331 pp512 via Vulkan RADV (b8933). 109B parameter model running on a mini PC; this artifact exceeds 24GB VRAM without offload
- Merged PR #1: vulkan-tools install check in setup.sh (thanks @ignasivt)
- Updated April price snapshot for Beelink, Corsair, and GMKtec; superseded by the May 1 price audit above
- Added linux-firmware-20251125 source attribution and downgrade instructions
- Added Ubuntu 26.04 LTS note (Wayland-only, testing in progress)
- Ollama upgraded to 0.21.2: FA now enabled by default. Original Qwen3.6 via Ollama result was 45.5 t/s; superseded by the May 2026 controlled 50.5 t/s rerun.
- Ollama ROCm confirmed working on gfx1151 with
HSA_OVERRIDE_GFX_VERSION=11.5.1(Ollama 0.20.4). Benchmarked: 42.4 t/s tg vs Vulkan's 46.6 t/s (-9%). Vulkan still recommended for speed
Performance discoveries:
- llama.cpp b8298 to b8460 = +25% tg and +24% pp on MoE models (52 to 65 t/s tg, 868 to 1080 pp512)
- Key PRs: #19625 (FA refactor), #20551 (graphics queue), #20334 (GDN shader)
- +25% breaks down as ~14% generic (both backends got this) + ~11% Vulkan-specific
- Dense models show <2% change (already at bandwidth ceiling)
- RADV now beats AMDVLK on both pp AND tg with latest build (old AMDVLK tg advantage gone)
- Exceeded theoretical tg ceiling: measured 65 t/s vs calculated max of ~57 t/s. The standard formula (bandwidth / active_model_size) underestimates MoE performance because it ignores caching and memory access optimizations in newer llama.cpp builds. The real ceiling is a moving target.
- RADV now beats ROCm on both pp (1080 vs 1047) and tg (65 vs 55) on same b8460 build
- ROCm works on kernel 6.19.4 with
HSA_OVERRIDE_GFX_VERSION=11.5.1+HSA_ENABLE_SDMA=0 - ROCm b8460 got +14% tg from generic improvements (47.87 to 54.67)
- Batch/ubatch sweep: default 512 is optimal, no tuning headroom left
New benchmarks:
- Llama 3.1 70B (4.8 t/s, 94% of theoretical ceiling, doesn't fit on RTX 4090)
- Qwen3-Coder-30B UD-Q4_K_XL (87 t/s tg via RADV, superseded by the May 2026 97 t/s controlled rerun)
- Qwen3-0.6B (266 t/s tg, 13,112 pp512)
- Extended context scaling (pp flat from 512 to 8K, only 3% drop)
Beginner content:
- Ollama vs llama.cpp FAQ with browser analogy and llama-server setup
- Model recommendation guide (10 use cases)
- Cost comparison (local vs cloud with break-even analysis)
- Buying guide (7 systems with a historical March 2026 price snapshot and Beelink board/NIC revision guidance)
- Glossary (20+ terms for beginners)
- FAQ (8 common questions)
- Use cases (Claude Code, Cursor, RAG, image gen, TTS)
- Windows vs Linux comparison
Infrastructure:
- One-command setup script (
setup.sh) - Auto-update script for llama.cpp (
update-and-build.sh) - CONTRIBUTING.md and initial GitHub issue templates; expanded in later May updates for benchmarks, power reports, model requests, bugs, suggestions, and impersonation/security reports
- Historical launch release v1.0.0; current release history is preserved on the GitHub Releases page.
- 19 topics for discoverability
- GitHub stars + last-commit badges
Fixes:
- At that time, prices were verified against current retail (March 2026 snapshot)
- Historical DGX Spark wording called the comparison apples-to-apples; September correction: exact artifact/backend/cache/workload matching was not established (see Hardware Comparison)
- Fixed 12 outdated "ROCm broken on 6.19.x" references
- A low BIOS UMA reserve is mandatory, not just speed-neutral: use 512MB if available, but 2GB is fine when that is the vendor minimum
- Vulkan Driver Comparison updated with b8460 data
- RADV_PERFTEST env vars (cswave32, nogttspill) tested and found to be -10% slower. Don't use.
- Posted findings on llama.cpp Vulkan discussion
- Complete rewrite with live benchmarks on current system
- Added: Kernel 6.19.x ROCm fix (HSA_OVERRIDE_GFX_VERSION=11.5.1)
- Added: Mesa 26.0.2 results (+4-5% tg improvement over 26.0.1)
- Added: qwen3-coder:30b-a3b-q8_0 benchmarks (51.4 t/s -- fastest model in that initial run)
- Added: Long context performance data from lhl (Vulkan vs ROCm at 32K)
- Added: rocWMMA status update (upstream broken, lhl's tuned branch works)
- Added: vLLM setup and known issues
- Added: RDMA clustering section
- Added: Kernel/ROCm compatibility matrix
- Added: linux-firmware-20251125 warning
- Added: LLVM compiler regression workaround
- Added: Qwen3.5 ROCm hang bug (ROCm #6027)
- Added: Backend decision guide
- Added: Testing checklist
- Added: Collapsible troubleshooting sections
- Updated: ROCm HIP works on kernel 6.19.4 with HSA override (even +6% faster pp than 6.18.14)
- Updated: All benchmark numbers re-measured
- Updated: Replaced
nanoinstructions withteefor copy-paste ready commands - Corrected: rocWMMA is no longer blanket "don't use" -- lhl's tuned branch is best for long context
- Historical benchmark correction:
iommu=ptdid not improve the measured memory-read result; later buyer guidance separated the fasteramd_iommu=offdesktop profile from NPU and mobile-suspend requirements
- Basic setup guide based on pablo-ross' GMKtec guide
- Ollama Vulkan configuration
- ROCm container setup
These are the highest-value tests to add next, because they answer practical buyer/setup questions that current evidence only partially covers:
- Lemonade/FastFlowLM NPU on Linux: local preflight shows
amdxdnaand/dev/accel/accel0, but XRT/FastFlowLM are not installed. Next step is a separate NPU lane: install XRT/FastFlowLM, reboot, runflm validate, then test small Qwen/Gemma rows for speed and power. - Same-machine Windows native Vulkan/Ollama/LM Studio vs Linux: Windows LM Studio and WSL2/HIP now have community baselines, but the most useful beginner answer is still a same-machine native Windows app result versus native Linux Vulkan/RADV with the same model and benchmark shape.
- More GMKtec/Bosgame/Framework native Linux reproductions: The first GMKtec native result matched within about 2%; more vendors turn the guide from one-machine evidence into a platform map.
- DeepSeek V4 Flash current route: the pinned 90.86GB ordinary
UD-IQ2_XXSGGUF now loads and generates directly on official b10034 at 13.27 tg128 and passes a basic deterministic check. The smaller 46.98GiB REAP route still needs its separate ds4 runtime; future work should compare quality/runtime tradeoffs rather than repeat the resolved ordinary-GGUF load test. - Tokens per watt with wall-power data: Fail-Safe supplied valuable Corsair wall-power telemetry, and this guide now has Beelink amdgpu PPT telemetry. A Beelink wall-meter run would make the efficiency story publishable.
- NPU/iGPU telemetry tooling:
xdna-topand similar tools could make NPU-sidecar and iGPU contention claims easier to verify, but should be documented as instrumentation until they produce measured model rows. - Lucebox / DFlash / PFlash: highest-upside experimental route for 27B long-prompt + generation workloads, but the old preflight lacked a developer toolchain; an isolated HIP toolchain now built the v0.4.1 controls, but this does not qualify Lucebox/DFlash; older rocWMMA design notes are historical after the 2026 upstream removal.
- vLLM/AWQ/DFlash throughput: keep this experimental until it has a reproducible OpenAI-compatible server row that competes with
llama-server/Ollama for a real use case. Plain AWQ smoke works, but it is not the fastest default. - Future Strix Halo successors: Gorgon Halo / Ryzen AI Max 400 and later Medusa Halo / Ryzen AI Max 500 should be treated as future comparison targets, not current setup advice.
This project is licensed under the MIT License. See LICENSE.
Found this guide useful? Give it a star on GitHub -- it helps other Strix Halo owners find it. Found something wrong? Open an issue.