Tencent DSpine: A Parallel Drafter That Beats DFlash on Qwen3-8B

Related: speculative decoding is becoming standard for big open models. See how Nvidia shipped a DFlash 2 draft model for Muse Spark Open and how DFlash 2 runs on an M5 Max with MLX.

Tencent has released DSpine, a drafter for speculative decoding that it says is faster than the DFlash family of parallel drafters. The checkpoint, tencent/Qwen3-8B-DSpine-b16, is on Hugging Face under Apache 2.0 with inference code, and the paper, "Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding", was submitted on September 28, 2026 and revised on October 10.

If you serve Qwen3-8B, or care how the draft-and-verify trick keeps getting faster, this is a drop-in experiment with a clear baseline. This post explains the idea in plain language, the numbers, how to try it with SGLang, and where the claims need caution.

TL;DR

What is speculative decoding, and what does DSpine change?

A large language model normally produces one token per forward pass. Speculative decoding speeds this up by letting a small, fast drafter guess several tokens ahead, then having the big target model check all the guesses in a single pass. Tokens that match are accepted for free, so the average number of tokens accepted per target pass (the "acceptance length") sets the speedup. For the basics, our explainer on tokens per second shows why this matters for cost.

Early drafters worked sequentially, which keeps guesses consistent but is slow. Newer drafters such as DFlash draft a whole block in parallel. The catch, as the DSpine abstract puts it, is that "independent token selection can produce inconsistent continuations that shorten the accepted prefix": if position 5 does not know what was guessed at position 4, the chain breaks and the target rejects the rest.

A fast draft step running ahead of a slower verification step, the core of speculative decoding in DSpine

DSpine's idea is to let positions talk to each other through depth rather than through time. At every layer of the drafter, a gated "adjacent injection" writes each predecessor's predicted feature into its successor. The causal chain unfolds over the layers, "while all positions are still drafted in parallel," in the words of the model card. The authors found in DFlash that early positions already form recoverable predictions in shallow layers, and that accurate predecessors help more when they enter earlier, so they inject throughout the backbone instead of leaving conditioning to a lightweight module at the end.

Two supporting pieces: a shared "transfer space" built from the target model's output embeddings, and layer-wise supervision so predictions form in shallow layers. Fused kernels and a transition cache keep it efficient inside SGLang.

What do the numbers say?

From the paper abstract: across seven math, code and chat benchmarks, DSpine reached the longest acceptance length at both temperatures on Qwen3-4B and Qwen3-8B. At temperature zero on Qwen3-8B the seven-benchmark mean rose from DFlash's 3.77 to 4.82, a 27.8% gain, and in SGLang serving tests it delivered 23.3% higher throughput than DFlash on average.

The Hugging Face model card gives a different table, where DSpine, DSpark and DFlash2 were all retrained on Open-PerfectBlend with the same configuration. Acceptance lengths at temperature zero:

Read this carefully. Against the strongest retrained baseline, DFlash2, DSpine's gain is about 4% on average (7.83 versus 7.55), which is real but modest. The headline 27.8% is against DFlash, an older baseline, and uses different numbers from the card. At temperature one, the gap to DFlash2 is similar (6.98 versus 6.53). Acceptance length is also not wall-clock speed: the paper's separate 23.3% throughput figure is the better guide for serving, and it is measured against DFlash, not DFlash2.

How do you try it?

The model card gives the recipe. Install the repo, then launch SGLang with the new speculative algorithm:

git clone https://github.com/tencent/dspine.git
cd dspine
pip install -e .
sglang serve --model-path Qwen/Qwen3-8B --speculative-algorithm DSPINE --speculative-draft-model-path <dspine-model-path> --speculative-num-draft-tokens 16 --dtype bfloat16 --attention-backend fa3

The card lists further flags used for the paper's results: page size 1, static memory fraction 0.65, an 8,192-token context, up to 64 running requests and CUDA-graph batch sizes from 1 to 64. Those match the settings behind the reported throughput, so copy them if you want comparable numbers. Note that the attention backend flag shown is for Hopper-class GPUs, and the setup assumes SGLang with the DSpine integration rather than vLLM or llama.cpp. Our coverage of llama.cpp speculative decoding on Apple silicon shows the local-runtime side of the same idea.

How should you benchmark a drafter on your own workload?

Published acceptance lengths come from tidy benchmarks such as GSM8K and HumanEval, where answers are short, structured and predictable. Production traffic is messier, and the gap between a benchmark and your logs is where speculative decoding wins or loses. A sensible evaluation has four steps.

First, sample real prompts. Pull a few hundred requests from your own logs, including the long, rambling ones, and replay them with a fixed seed. Chat-style traffic is the hardest case: in the model card table, MT-Bench and Arena-Hard acceptance lengths are roughly half those of GSM8K for every method, which tells you that open-ended text is harder to guess than arithmetic.

Second, measure the right thing. Acceptance length is a proxy. What you pay for is tokens per second at your batch size, and time to first token. Drafting adds work, so at high concurrency the benefit shrinks as the GPU becomes compute-bound. The paper's throughput test and the card's setup list 64 maximum running requests, so test at the concurrency you actually run, from one request up to your peak.

Third, compare against the drafter you already use. If you are on no speculative decoding at all, any of these drafters is a large win and the choice among them is a second-order question. If you already run DFlash or DFlash2, the realistic gain from DSpine is the smaller one in the card, not the headline.

Fourth, check quality and failure modes. Speculative decoding with exact verification should not change outputs, but implementations have bugs, and a new SGLang algorithm flag deserves a regression test on a fixed set of prompts at temperature zero before it reaches users. Also watch memory: a roughly 1B-parameter drafter adds weights and cache next to an 8B model, which matters on a 24 GB card.

This discipline is the same one we recommend for any new inference trick, as in our Magnitude engine claim check: reproduce on your hardware, with your prompts, before you quote a speedup to anyone.

Who should care?

• Teams serving Qwen3-8B at volume, where a 1B drafter and a 20% throughput gain translate directly into GPU hours.
• Inference engineers comparing drafters: DSpine adds a fourth data point next to DFlash, DSpark and DFlash2, and our speculative decoding guide covering DSpark.
• Researchers interested in conditioning through depth, since the idea may transfer to other parallel generators, including the diffusion-style approaches we covered in next concept modeling.

If you run models on a single consumer GPU, the benefit depends on your runtime adopting the algorithm, so watch for ports.

Limits and open questions

1. Training code is not out. The roadmap lists it as pending, so you cannot train a drafter for another target model yet.
2. One target model. Only Qwen3-8B has a released checkpoint, and a Qwen3.8-27B drafter is only a roadmap item. Large MoE targets such as Beam use different drafters.
3. Baseline choice matters. As shown, the improvement over the newest baseline is smaller than the headline.
4. Self-reported. Both the paper and the card are from the authors. We have not reproduced the numbers.
5. Serving stack lock-in. The speedup relies on fused kernels and a transition cache inside SGLang.
6. Temperature sensitivity. Gains hold at temperature one in the card, but acceptance drops for every method, so sampling-heavy workloads will see smaller speedups.

What this means for what you build or pay

Speculative decoding is one of the few inference tricks that cuts latency without changing quality, and each new drafter nudges the cost per token down. If you serve Qwen3-8B on SGLang, a one-afternoon A/B against your current drafter is cheap: measure tokens per second and acceptance length on your own prompts, not on GSM8K. If you serve something else, there is nothing to download, but the technique is worth tracking, because the same pattern, a small parallel drafter tuned to one target, is now being shipped for much larger models.

Related reading

• Nvidia DFlash 2 draft model for Muse Spark Open
• DFlash 2 on MLX and the M5 Max
• DeepSeek DSpark speculative decoding guide
• What is TPS? Tokens per second explained
• llama.cpp speculative decoding on M3 Ultra
• Reflection Beam open weights

Sources: DSpine model card · arXiv 2609.36173

Accurate as of October 16, 2026. All performance numbers are reported by the authors. The GitHub repository linked from the model card was not reachable when we checked.