Qwen 3.6 27B Local Dev Guide: llama.cpp, OpenCode, and Why Dense Beats MoE
Quesma’s Piotr Migdał benchmarks Qwen 3.6 27B on Apple Silicon — 32 tok/s with MTP, OpenCode minesweeper wins, AA score 37. llama.cpp setup, vs 35B MoE and Gemma 4 31B.
Update — July 19, 2026: Alibaba announced Qwen 3.8 (2.4T params, open-weight soon, Max-Preview on Token Plan now). Qwen 3.6 27B remains the practical local tier until smaller 3.8 distillations ship.
Update — July 16, 2026: Google shipped a Gemma 4 community update (FA4, tool calling, vision). Qwen 3.6 27B still leads text-only local coding in community tests; Gemma 4 wins multimodal and edge — see the comparison table in the new post.
June 29, 2026 — Hacker News front page:Quesma co-founder Piotr Migdał published a hands-on verdict: Qwen 3.6 27B dense is the first local model that feels like general intelligence worth daily use — not a toy quant, not a MoE that races past instructions.
Update — July 3, 2026 (explainx.ai field test): After publishing this guide, @goyashy tested Qwen 3.6 27B on RTX 4070 + Ollama + OpenCode for routine daily tasks — GPU usage jumps to 90%+ on every prompt. Workable, but 12GB VRAM saturates fast. We folded this live feedback into the sections below (Ollama path, RTX tuning, honest expectations).
35B A3B — faster generation, but single index.html, ignored “create a package”
That matches broader Mac local LLM guidance: MoE locals can loop, confuse identity, or skip structure — dense often wins for coding.
HN consensus echoed the post: “punches above its weight” — see threads like Will it Mythos? comparing local Qwen to past frontier tiers.
Smoke tests — from quantum poems to candle shops
Migdał’s eval ladder (beyond benchmarks):
Constrained writing
Quantum mechanics chat and an 8-line poem mixing Zouk dance and quantum physics — reasoning and rhyme deliberation both coherent. A year ago that quality needed GPT-4.5-class cloud spend (cf. Simon Willison’s “penguins on a bicycle” smoke test pattern).
Real-ish product work
Maciej Cielecki’s candle-shop prompt (AI Tinkerers Warsaw) via OpenCode — a few minutes → reactive landing page, sensible defaults, single short prompt. Not frontier-shocking in 2026, but a practical job that worked.
Simon Willison’s visual smoke test
Willison ran the same penguin on a bicycle prompt against 35B A3B then 27B — useful third-party replication anchor linked from the Quesma post.
explainx.ai read: treat these as integration tests, not SWE-bench. They predict “will my harness produce a repo I can npm install” better than MMLU alone.
llama.cpp setup — copy-paste
Foundation: New to the engine? Read what is llama.cpp? — install, GGUF, llama-cli vs llama-server, and flag reference before model-specific tuning below.
Server mode (agents + API)
Recommended quant: unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 — 8-bit, multi-token prediction support. See quantization guide for Q4/Q8 trade-offs.
~32 tok/s with MTP on 27B is within typical cloud API decode range for many products
llama.cpp beat MLX in these runs (~95% GPU utilization)
35B A3B is 3× faster — but Migdál prefers ⅓ the tokens at higher quality
DwarfStar4 (aggressive DS V4 Flash quant) matches ~33 tok/s but uses 103 GB RAM — poor laptop fit
HN RTX 5090 datapoint (gfosco): Q6_K + Q4_0 KV → ~50 tok/s at 123k context, ~28/32GB VRAM through LM Studio.
Both Qwen variants fit 48GB Apple Silicon at 8-bit; 4-bit under 18GB targets 32GB Macs. Consumer Nvidia needs aggressive quants but can exceed Mac tok/s — see Mac vs dedicated GPU guide.
Thermal warning: Migdał literally thermal-imaged the machine — 27B local is hot. Plan cooling and power.
We published this guide June 29. On July 3, explainx.ai founder @goyashy pulled Qwen 3.6 27B after reading it — testing whether it could replace basic day-to-day cloud models on consumer hardware:
Setup
Detail
GPU
NVIDIA RTX 4070 (12GB VRAM)
Runtime
Ollama (lowest-friction path vs llama.cpp in the Quesma post)
Harness
OpenCode for routine agent tasks
Observation
GPU usage spikes to 90%+ on every prompt drop
Verdict
Works for routine tasks; VRAM headroom is the bottleneck
This is exactly the kind of feedback we wanted when we wrote the guide — not benchmark theater, but "can I actually run my Tuesday workflow on this?"
What 90% GPU usage means on RTX 4070
12GB VRAM is tight for 27B dense at usable quants:
Q8 — often won't fit with context + KV cache; expect OOM or aggressive swapping
Q4_K_M — realistic default for 4070-class cards
Context — start at 8k–16k, not 64k, until you profile VRAM
Concurrent apps — close browser GPU tabs; 90% saturation leaves no margin
If Ollama saturates and throttles, switch to llama.cpp with partial CPU offload (-ngl tuned to leave ~1GB headroom) — the Quesma path in this guide.
Ollama quick start (4070 path)
bash
ollama pull qwen3.6:27b
ollama run qwen3.6:27b
Point OpenCode at Ollama's OpenAI-compatible endpoint:
Next hardware step:GLM-5.2 via Unsloth — not laptop-friendly, but company-budget realistic for closer-to-frontier local.
Longer horizon: models may split intelligence from factual knowledge — raw weights for reasoning, tools/MCP for facts — shrinking local hardware needs. See agent skills guide and MCP overview.
Who should run Qwen 3.6 27B locally
Strong fit:
48GB+ Apple Silicon or 24GB+ Nvidia with acceptable quants
OpenCode / Pi / Hermes users wanting offline or private agent loops
Developers priced out of or blocked from frontier APIs post-Fable
Benchmarks and tok/s figures from Quesma's M5 Max tests and cited HN reports as of June 29–30, 2026. RTX 4070 Ollama field test added July 3, 2026 from @goyashy live testing. Verify on your hardware before production routing.
