Paper review · November 17, 2026
MMFlash: Faster Speculative Decoding for Vision-Language Models
The authors build a diffusion drafter that gives more training weight to tokens that depend heavily on the image, speeding up VLM decoding by up to 3.58x with no loss in output.
Problem
Diffusion drafters such as DFlash propose an entire block of tokens in a single forward pass, but they are mainly tuned for text tasks. According to the authors' analysis, in VLMs, the more a token depends on the image, the lower its draft acceptance rate. This gap remains even when training on multimodal data.
Method
Per-token visual dependence is defined as the KL divergence between the target model's next-token distributions computed with and without the image. This value is computed once offline. A grounding score is formed by multiplying the dependence quantile by the target's confidence (exp(-entropy)). DFlash's position-based weights are replaced with chained weights that reflect the probability of surviving verification. On top of this, a bonus equal to 'grounding score × probability that the drafter is still wrong' is added, concentrating training on under-learned visual tokens. For long video inputs, sparse attention that keeps only attention sinks, visual tokens at fixed intervals, all text tokens and a local window cuts a 40K+ token context to about 1.6K.
Results
- The analysis found an acceptance rate of 46% for tokens in the bottom 25% of visual dependence (Q1) and 22% for the top 25% (Q4). 44% of first rejection points within a block fell in Q4 (7% in Q1).
- Across seven image benchmarks, the average greedy speedup was 3.00x for Qwen3.5-4B and 3.05x for 9B. This is 10.3–10.9% higher than DFlash (2.72x, 2.75x) and well ahead of EAGLE-3 (1.58x, 1.68x).
- The maximum speedup was 3.58x on ChartQA with Qwen3.5-9B (DFlash: 3.20x). At temperature 1, the 4B model still averaged 2.54x, higher than DFlash (2.38x).
- On three long-video benchmarks, sparse attention and a small amount of video fine-tuning yielded an average of 2.33–2.71x, 11.2–23.2% faster than DFlash.
Limitations
Evaluation focuses mainly on the Qwen3.5 family (4B and 9B). Generalization to other model families and larger scales is mentioned in the text, but detailed numbers could not be confirmed within the provided range. Preprocessing is costly because the target model must be run twice over the entire training set, with and without the image. Additional fine-tuning is needed to get video performance.
Why it matters
VLM serving costs are rising because of long reasoning responses. This paper shows numerically where text-oriented speculative decoding loses out when carried over unchanged, and pushes lossless speedup further by changing only the training objective. Measurements were taken in SGLang, and the code and models are released.
Key terms
- Speculative Decoding
- A technique in which a small drafter proposes several tokens ahead and a large target model verifies them all at once. It speeds up generation without changing the output distribution.
- Diffusion Drafter
- A drafter that uses block diffusion to fill an entire draft block in a single forward pass. With no sequential generation cost, it can use a deeper model.
- Acceptance length (τ)
- The average number of tokens the target accepts per verification step. The longer it is, the greater the speedup.
- KL divergence
- A measure of how different two probability distributions are. Here it is used to measure how much the image changes next-token prediction.
Original
Authors: Sicheng Feng, Zigeng Chen, Song Wang, Gongfan Fang, Xinchao Wang · Posted 2026-11-13 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.