DFlash-MLX Brings Lossless Speculative Decoding to Apple Silicon — Up to ~189 tok/s on M5 Max

Headlines this week claimed Apple Silicon local inference got a flat 4.6× speed jump from a new MLX port of DFlash — block-diffusion speculative decoding that drafts 16 tokens per cycle and verifies them in one target forward pass. The project is real: dflash-mlx (bstnxbt, Apache-2.0, 760+ GitHub stars as of August 19, 2026) ships lossless greedy verification on MLX for Apple Silicon, with an OpenAI-compatible server, prefix cache, and published M5 Max benchmarks.

The correction worth leading with: speedup is not one number. On M5 Max 64 GB, Qwen3.5-4B reaches 188.7 tok/s (3.49× over baseline) at 2048 tokens, Qwen3.5-9B hits 113 tok/s (3.65×), and the ~70 tok/s figure most feeds cite maps to Qwen3.5-27B-4bit at 70.2 tok/s — roughly 2.12×, not 4.6×. MoE models that were already fast without speculation gain less. That matters if you're sizing a Mac for local open-weight work while RAM prices stay elevated.

TL;DR

What DFlash and dflash-mlx actually do

DFlash (Chen, Liang, Liu — February 2026) replaces token-by-token autoregressive decoding with block diffusion: a small drafter (~1B parameters) predicts 16 tokens in parallel, then the full target model verifies the entire block in one forward pass. Greedy acceptance keeps the longest correct prefix and rolls back the rest.

dflash-mlx implements that loop on stock MLX with targeted Metal kernels for rollback on GatedDeltaNet hybrids (Qwen3.5/Qwen3.6), verify-specialized int4 quantized matmuls on M5-class GPUs, and an L1+L2 prefix cache inspired by tiered KV work in the MLX ecosystem. The README is explicit about the quality contract:

"Lossless: every emitted token is the target model's greedy argmax at verification time."

Output can still differ slightly from pure AR because of MLX dispatch edge cases, but no unverified token is emitted — this is not a lossy draft shortcut.

Meta adopted the same DFlash family for Muse Glimmer's optional drafter (16-token blocks, quantized companion model). dflash-mlx generalizes the technique across the Qwen and Gemma MLX ports many Mac builders already run.

M5 Max benchmarks — corrected, not inflated

All figures below come from the dflash-mlx README benchmark table: Apple M5 Max, 64 GB, MLX 0.31.1, median of 3 sequential runs, 60 s cooldown, greedy decoding, --no-eos. Generation used a fixed math-reasoning prompt at varying max_tokens.

Dense and MoE Qwen targets (2048 tokens — the mid-context sweet spot)

Three patterns jump out:

1. Smaller dense models get the biggest multiples — 9B at 1024 tokens peaks at 4.37× in the full table (135.3 vs 31.0 tok/s).
2. The "~70 tok/s" headline is the 27B-4bit row, not the 4B or 9B peak — and its speedup is ~2.1×, not 4.6×.
3. MoE models with strong baselines gain less in relative terms — Qwen3.5-35B-A3B-4bit was already at 141 tok/s without DFlash; speculation adds throughput but not a 4× wall.

Context length erodes speedup on large quants

Long generations on memory-bandwidth-bound 27B quants still benefit, but the multiplier compresses as context grows — plan benchmarks at the token counts you actually generate.

Meta Glimmer — same technique, different stack

Meta's official Muse Glimmer ExecuTorch model card reports DFlash on K-Quant-17GB with batch size 1, greedy decoding, averaged across a diverse prompt set:

Glimmer's ~29B agent model and ExecuTorch runtime are not the same workload as dflash-mlx's Qwen3.5 table — but the 1.8× on M5 Max is a useful sanity check that speculative decoding on Apple Silicon often lands in the 1.5–3× band for larger models, not a flat 4.6× everywhere.

Reproduce any dflash-mlx row locally with:

pip install "dflash-mlx[bench]"

dflash benchmark \
  --model Qwen/Qwen3.5-9B \
  --max-tokens 2048 \
  --repeat 3 \
  --cooldown 60 \
  --no-eos

JSON artifacts land under benchmark/results/ in the repo.

Why feeds overstate the multiplier

Aggregator headlines tend to pick the largest ratio in the table (9B @ 1024 → 4.37×) or round "up to 4×" into "4.6× for everyone." The primary README never claims 4.6× as a universal Apple Silicon speedup — the numbers are model-, quant-, and context-specific.

If you're choosing hardware using explainx.ai's Mac vs dedicated GPU local LLM economics or picking models from the top open-weight laptop list, benchmark the exact model and context you'll run — not a viral ratio.

Install and run today

pip install dflash-mlx

# One-shot — draft auto-resolved for registered targets
dflash generate --model Qwen/Qwen3.5-9B --prompt "Explain speculative decoding in three sentences."

# OpenAI-compatible server (OpenCode, Continue, Open WebUI, LM Studio)
dflash serve \
  --model mlx-community/Qwen3.6-27B-4bit \
  --draft z-lab/Qwen3.6-27B-DFlash \
  --port 8000

# Environment check
dflash doctor
dflash models

Point OpenCode at http://127.0.0.1:8000/v1:

{
  "provider": {
    "dflash": {
      "name": "dflash-mlx (local)",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://127.0.0.1:8000/v1",
        "apiKey": "local"
      },
      "models": {
        "qwen3.6-27b-4bit": { "name": "Qwen3.6 27B + DFlash" }
      }
    }
  },
  "model": "dflash/qwen3.6-27b-4bit"
}

Live throughput and acceptance metrics: curl http://127.0.0.1:8000/metrics while a request is in flight.

Useful flags:

Default server policy enables prefix cache, L2 snapshots, and a 4 GB MLX cache limit — override only when you know why.

What people are asking

Do I need a separate drafter download?

For registered Qwen3.5/Qwen3.6/Gemma4 targets, dflash resolves the z-lab DFlash draft automatically. Draft weights default to w4 quantization in memory; pass --draft-quant none for bf16 A/B tests.

Will this run on M4 Pro / M2 Max?

Published numbers are M5 Max 64 GB only. Older Apple GPUs use steel simdgroup-MMA fallbacks instead of M5's Metal 4 NAX path for verify QMM — expect different absolutes. Run dflash benchmark on your machine; don't extrapolate from M5 rows.

Does it help coding agents?

Throughput helps any long decode — agent loops, OpenCode sessions, tool-call chains. The README notes a roadmap item for target-only AR fallback when speculative surplus goes negative on structured tool outputs; test your model's tool-call format before assuming DFlash helps there.

How much extra memory does the drafter cost?

Roughly a ~1B-parameter companion plus cache snapshots if prefix cache is on. On tight 32 GB Macs running 27B quants, verify headroom with dflash serve --diagnostics basic and watch rss_gb on /metrics — same discipline as Kimi K3 desktop prep on unified memory.

Is this better than llama.cpp MTP?

Different mechanism. llama.cpp draft-MTP uses the model's own multi-token prediction head; DFlash uses a separate block-diffusion drafter. dflash-mlx is MLX-native — if your stack is already MLX (not GGUF/llama-server), this is the path that integrates without re-quantizing to GGUF.

Honest limitations (from the repo, not the hype)

• Architecture coverage is narrow — optimized for Qwen3.5/Qwen3.6 GatedDeltaNet hybrids and Gemma4; pure Qwen3 attention targets work but skip tape-replay rollback.
• No Windows / CUDA path — Apple Silicon + MLX only; NVIDIA builders stay on vLLM, llama.cpp, or Glimmer's ExecuTorch/CUDA exports.
• Long-context gains fade on large quants (see 27B-4bit @ 8192 → 1.34×).
• Prefix cache + positional sparse prefill are powerful but easy to misconfigure — incoherent token subsets collapse draft acceptance per README.
• Early project — roadmap lists more backends, tool-call auto-fallback, and sustained long-context acceptance work; pin versions for anything production-shaped.

Summary

dflash-mlx brings lossless DFlash speculative decoding to MLX on Apple Silicon with a practical CLI and OpenAI-compatible server. On M5 Max 64 GB, expect up to ~189 tok/s on Qwen3.5-4B (3.5×), ~113 tok/s on 9B (3.7×), and ~70 tok/s on Qwen3.5-27B-4bit (~2.1×) — the configuration behind most "~70 tok/s" headlines. MoE models already above 140 tok/s baseline see smaller relative gains (~1.8×). Correct any "4.6× everywhere" claim: speedup varies by model size, quantization, and context, and Meta's Glimmer reports 1.8× on the same M5 Max with its own DFlash drafter.

If unified-memory Macs are your local platform, this is worth benchmarking against your actual model — especially while RAM remains the dominant build cost and software speedups partially offset hardware sticker shock.

Related on explainx.ai

• RAM prices up 500% — what that means for local AI builds
• How to run open-source models locally with OpenCode
• Kimi K3 — run locally on open-weight desktop hardware
• Top 10 open-weight models you can run on a laptop
• MacBook vs dedicated GPU for local LLM economics
• OrcaRouter Qwen3.8-27B MLX build for Mac
• Qwen 3.6 27B — llama.cpp + OpenCode deep dive
• What is llama.cpp?
• Sentence Transformers v6 — ColBERT multi-vector RAG
• OpenAI pauses frontier RL training

Primary sources: bstnxbt/dflash-mlx on GitHub · DFlash paper (arXiv 2602.06036) · Meta Muse Glimmer ExecuTorch model card · Meta AI — Introducing Muse Glimmer

Benchmark figures, model registry, and CLI flags reflect the dflash-mlx README and repository state as of August 19, 2026. Star count and acceptance rates change with releases — run dflash benchmark on your hardware before sizing production workflows. Meta Glimmer numbers are vendor-reported on ExecuTorch, not independently reproduced by explainx.ai.