DScout Speeds Up Qwen3-8B by 4.14× With a Diffusion Drafter

Researchers have introduced DScout, a method for training and tuning a diffusion drafter for speculative decoding. Across six benchmarks, it delivers an average speedup of 3.84× on Qwen3-4B and 4.14× on Qwen3-8B relative to standard autoregressive generation. Against the best existing baseline, the gain is 9.1% and 7.1%, respectively. A separate inference-time configuration policy adds up to 9.75% on top of DScout's fixed settings.

Where speculative decoding loses speed

The scheme is familiar: a small drafter proposes several tokens, and the target model verifies them in parallel. Diffusion drafters go a step further and generate a whole block of tokens at once rather than one at a time. The logic is simple: the more proposals and the higher the mean accepted length (MAT), the faster the system runs. The authors show that this logic breaks down in two places.

First, verification accepts only a contiguous prefix. If the token at the second position is rejected, correct predictions at the fifth and sixth positions go to waste. A standard loss that averages error across all positions does not account for this. Second, a higher MAT does not guarantee higher throughput. A deeper drafter and a longer block lengthen every round, and the gain in accepted tokens can be entirely eaten up by latency.

Two factorizations instead of one metric

The first factorization applies to training. DScout splits the positional loss into three components: whether a position is included in training (inclusion), its weight (weighting), and the prediction error (prediction loss). This breakdown suggested three concrete techniques: mixed-horizon training, intra-block unmasking, and a loss weighted by acceptance probability. Together, they align the training signal with how prefix verification works and raise MAT.

The second factorization applies to inference. Throughput is written as the ratio of MAT to the latency of a single round. This points to the parameters worth controlling: the target-conditioning depth, the depth of the drafter itself, and the block size at inference. Each workload gets its own configuration rather than one setting for everyone.

Numbers from the paper

• Qwen3-4B: average speedup of 3.84× over autoregressive decoding, 9.1% better than the strongest baseline.
• Qwen3-8B: average speedup of 4.14×, 7.1% better.
• The evaluation covers six benchmarks.
• The configuration-selection policy adds up to 9.75% of extra speedup relative to DScout's fixed configuration.
• Testing was run across different models and at different concurrency levels.

What inference engineers should take from this

The main practical takeaway: MAT should not be treated as the target metric. If you are tuning speculative decoding in your own stack, measure tokens per second on real workloads and look at both factors, MAT and round latency. The optimal block size and drafter depth depend on the request profile and concurrency, so a single config for all modes most likely leaves a noticeable share of speed on the table. In the paper, that share is up to 9.75%.

There are limitations. The results are reported for Qwen3-4B and Qwen3-8B. Based on the description, it is unknown how the method will behave on larger models or on models with a different architecture. The gain over a strong baseline is 7–9%: this is an engineering improvement, not a change of class. On the other hand, it comes without modifying the target model, and the idea of training a drafter for prefix acceptance is applicable beyond diffusion-based schemes.