Qwen 3.6 27B Local Dev Guide: llama.cpp, OpenCode, and Why Dense Beats MoE

Update — July 19, 2026: Alibaba announced Qwen 3.8 (2.4T params, open-weight soon, Max-Preview on Token Plan now). Qwen 3.6 27B remains the practical local tier until smaller 3.8 distillations ship.

Update — July 16, 2026: Google shipped a Gemma 4 community update (FA4, tool calling, vision). Qwen 3.6 27B still leads text-only local coding in community tests; Gemma 4 wins multimodal and edge — see the comparison table in the new post.

June 29, 2026 — Hacker News front page: Quesma co-founder Piotr Migdał published a hands-on verdict: Qwen 3.6 27B dense is the first local model that feels like general intelligence worth daily use — not a toy quant, not a MoE that races past instructions.

Update — July 3, 2026 (explainx.ai field test): After publishing this guide, @goyashy tested Qwen 3.6 27B on RTX 4070 + Ollama + OpenCode for routine daily tasks — GPU usage jumps to 90%+ on every prompt. Workable, but 12GB VRAM saturates fast. We folded this live feedback into the sections below (Ollama path, RTX tuning, honest expectations).

The post landed amid Fable 5 restoration July 1, GLM-5.2 open-weight momentum, and recurring HN threads on Mac vs Nvidia for local LLMs. Migdał's angle is narrower and more actionable: pick dense 27B over MoE 35B, run llama.cpp with MTP, wire OpenCode, measure ~32 tok/s on M5 Max, and accept the heat.

TL;DR — what people are asking

Why dense 27B beats MoE 35B A3B (in practice)

Alibaba ships two local-friendly Qwen 3.6 shapes:

The minesweeper test in OpenCode crystallized it:

• 27B — hexagonal minesweeper, pnpm package, first prompt, proper Node layout
• 35B A3B — faster generation, but single index.html, ignored “create a package”

That matches broader Mac local LLM guidance: MoE locals can loop, confuse identity, or skip structure — dense often wins for coding.

HN consensus echoed the post: “punches above its weight” — see threads like Will it Mythos? comparing local Qwen to past frontier tiers.

Smoke tests — from quantum poems to candle shops

Migdał’s eval ladder (beyond benchmarks):

Constrained writing

Quantum mechanics chat and an 8-line poem mixing Zouk dance and quantum physics — reasoning and rhyme deliberation both coherent. A year ago that quality needed GPT-4.5-class cloud spend (cf. Simon Willison’s “penguins on a bicycle” smoke test pattern).

Real-ish product work

Maciej Cielecki’s candle-shop prompt (AI Tinkerers Warsaw) via OpenCode — a few minutes → reactive landing page, sensible defaults, single short prompt. Not frontier-shocking in 2026, but a practical job that worked.

Simon Willison’s visual smoke test

Willison ran the same penguin on a bicycle prompt against 35B A3B then 27B — useful third-party replication anchor linked from the Quesma post.

explainx.ai read: treat these as integration tests, not SWE-bench. They predict “will my harness produce a repo I can npm install” better than MMLU alone.

llama.cpp setup — copy-paste

Foundation: New to the engine? Read what is llama.cpp? — install, GGUF, llama-cli vs llama-server, and flag reference before model-specific tuning below.

Server mode (agents + API)

Recommended quant: unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 — 8-bit, multi-token prediction support. See quantization guide for Q4/Q8 trade-offs.

llama-server -hf unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 \
    --spec-type draft-mtp -ngl 999 -fa on -c 65536 --port 8080

Chat UI: http://127.0.0.1:8080

Local file instead of -hf:

llama-server -m ~/models/Qwen3.6-27B-Q8_0.gguf \
    --spec-type draft-mtp -ngl 999 -fa on -c 65536 --port 8080

Terminal-only chat

llama-cli -hf unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 \
    -ngl 999 -fa on -c 65536

Popular quants also from bartowski and other GGUF publishers on Hugging Face.

OpenCode config — local provider block

Add to ~/.config/opencode/opencode.jsonc:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "llama": {
      "name": "llama.cpp (local)",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://127.0.0.1:8080/v1",
        "apiKey": "local"
      },
      "models": {
        "qwen3.6-27b": { "name": "Qwen3.6-27B Q8 +MTP" }
      }
    }
  },
  "model": "llama/qwen3.6-27b"
}

Same server works for Pi, Hermes, or any OpenAI-compatible harness. Pillar guide: How to run open-source models locally in OpenCode — Ollama, LM Studio, and multi-provider tiering. For GLM and cloud fallbacks in one config, see How to Run GLM 5.2 on every harness.

Performance numbers — M5 Max 128GB

Migdał’s benchmarks (source notebook linked from Quesma post) on MacBook Pro Max M5, 128GB:

Takeaways:

• ~32 tok/s with MTP on 27B is within typical cloud API decode range for many products
• llama.cpp beat MLX in these runs (~95% GPU utilization)
• 35B A3B is 3× faster — but Migdál prefers ⅓ the tokens at higher quality
• DwarfStar4 (aggressive DS V4 Flash quant) matches ~33 tok/s but uses 103 GB RAM — poor laptop fit

HN RTX 5090 datapoint (gfosco): Q6_K + Q4_0 KV → ~50 tok/s at 123k context, ~28/32GB VRAM through LM Studio.

Both Qwen variants fit 48GB Apple Silicon at 8-bit; 4-bit under 18GB targets 32GB Macs. Consumer Nvidia needs aggressive quants but can exceed Mac tok/s — see Mac vs dedicated GPU guide.

Thermal warning: Migdał literally thermal-imaged the machine — 27B local is hot. Plan cooling and power.

Field report — RTX 4070 + Ollama + OpenCode (July 3, 2026)

We published this guide June 29. On July 3, explainx.ai founder @goyashy pulled Qwen 3.6 27B after reading it — testing whether it could replace basic day-to-day cloud models on consumer hardware:

This is exactly the kind of feedback we wanted when we wrote the guide — not benchmark theater, but "can I actually run my Tuesday workflow on this?"

What 90% GPU usage means on RTX 4070

12GB VRAM is tight for 27B dense at usable quants:

• Q8 — often won't fit with context + KV cache; expect OOM or aggressive swapping
• Q4_K_M — realistic default for 4070-class cards
• Context — start at 8k–16k, not 64k, until you profile VRAM
• Concurrent apps — close browser GPU tabs; 90% saturation leaves no margin

If Ollama saturates and throttles, switch to llama.cpp with partial CPU offload (-ngl tuned to leave ~1GB headroom) — the Quesma path in this guide.

Ollama quick start (4070 path)

ollama pull qwen3.6:27b
ollama run qwen3.6:27b

Point OpenCode at Ollama's OpenAI-compatible endpoint:

{
  "provider": {
    "ollama": {
      "name": "Ollama (local)",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://127.0.0.1:11434/v1",
        "apiKey": "ollama"
      },
      "models": {
        "qwen3.6-27b": { "name": "Qwen3.6 27B" }
      }
    }
  },
  "model": "ollama/qwen3.6-27b"
}

Monitor GPU during first session:

# Windows — watch utilization while OpenCode runs
nvidia-smi -l 1

Honest tiering after this test:

We will keep updating this section as more readers report real-world configs — reply on X with your setup if you test.

Benchmark grounding — Artificial Analysis tiers

Migdał maps Artificial Analysis Intelligence Index scores to era:

Caveats:

• Q8 Qwen barely hurts quality; 2–4 bit DwarfStar4 is not full V4 Flash
• Migdał’s impression: 27B ≈ or slightly beats DwarfStar4 within those quants; long-context projects may still favor DS4
• Gemma 4 on Cerebras is a cloud speed story — different from local 31B coding default

For Alibaba’s cloud/agent line, see Qwen 3.8-Max-Preview Token Plan (Jul 19, 2026), Qwen 3.7-Max agent frontier, and top Chinese AI companies.

Fable, subsidies, and what local buys you

Migdał closes with policy context explainx.ai readers already live:

• Fable 5 taken down — proprietary frontier access is policy-variable
• $100/mo consumer subs still imply massive inference subsidy — “use the discount while it lasts”
• Local weights — fine-tuneable, cannot be revoked, safe for proprietary code, offline, medical, or jurisdictional discomfort with US/China cloud

Tiered strategy (matches enterprise Fable alternatives):

Next hardware step: GLM-5.2 via Unsloth — not laptop-friendly, but company-budget realistic for closer-to-frontier local.

Longer horizon: models may split intelligence from factual knowledge — raw weights for reasoning, tools/MCP for facts — shrinking local hardware needs. See agent skills guide and MCP overview.

Who should run Qwen 3.6 27B locally

Strong fit:

• 48GB+ Apple Silicon or 24GB+ Nvidia with acceptable quants
• OpenCode / Pi / Hermes users wanting offline or private agent loops
• Developers priced out of or blocked from frontier APIs post-Fable
• Volume coding where 32 tok/s is enough (build personal local AI system)

Skip or cloud instead:

• Hard SWE-bench / multi-hour agent runs — use GLM-5.2 harness or cloud
• Multimodal local — Gemma 4 12B/31B instead
• 32GB Mac without quant discipline — start Q4 and short context

Related on explainx.ai

• OpenCode Desktop tabs — Jul 15, 2026 — session-per-tab shell; worktrees deferred
• Fix local LLM looping with samplers and sandboxing — DRY/top_n_sigma sampler fixes for Qwen 3.6 27B looping, plus 2×3090 vs MacBook concurrency benchmarks
• PrismML Bonsai 27B — 1-bit 3.9GB + HN harness reality (Jul 2026) — extreme low-bit Qwen3.6 compression vs GGUF; Gemma 4 12B QAT comparison
• Meta Pocket vibe-coded gizmos — consumer-side prompt-native apps (contrast with dev local stack)
• Mac vs dedicated GPU for local LLMs — hardware thread Qwen 3.6 appears in
• Build a personal local AI system — Ollama vs llama.cpp vs vLLM
• GLM-5.2 Unsloth local setup — next step up from Qwen 3.6
• GLM-5.2 on OpenCode and harnesses — shared config patterns
• Gemma 4 12B local guide — multimodal local alternative
• Fable 5 open-source alternatives — tiered inference
• Top Chinese AI companies — Qwen in Alibaba stack
• Model quantization guide — Q8 vs Q4 math

Source: Quesma — Qwen 3.6 27B sweet spot for local development (Piotr Migdał, June 29, 2026) · unsloth GGUF · OpenCode config schema

Benchmarks and tok/s figures from Quesma's M5 Max tests and cited HN reports as of June 29–30, 2026. RTX 4070 Ollama field test added July 3, 2026 from @goyashy live testing. Verify on your hardware before production routing.