← Key papers

Paper review · November 17, 2026

MMFlash: Faster Speculative Decoding for Vision-Language Models

Efficiency & infraMultimodal Advanced Upvotes 35 cs.CV National University of Singapore

The authors build a diffusion drafter that gives more training weight to tokens that depend heavily on the image, speeding up VLM decoding by up to 3.58x with no loss in output.

Problem

Diffusion drafters such as DFlash propose an entire block of tokens in a single forward pass, but they are mainly tuned for text tasks. According to the authors' analysis, in VLMs, the more a token depends on the image, the lower its draft acceptance rate. This gap remains even when training on multimodal data.

Method

Per-token visual dependence is defined as the KL divergence between the target model's next-token distributions computed with and without the image. This value is computed once offline. A grounding score is formed by multiplying the dependence quantile by the target's confidence (exp(-entropy)). DFlash's position-based weights are replaced with chained weights that reflect the probability of surviving verification. On top of this, a bonus equal to 'grounding score × probability that the drafter is still wrong' is added, concentrating training on under-learned visual tokens. For long video inputs, sparse attention that keeps only attention sinks, visual tokens at fixed intervals, all text tokens and a local window cuts a 40K+ token context to about 1.6K.

Results

Limitations

Evaluation focuses mainly on the Qwen3.5 family (4B and 9B). Generalization to other model families and larger scales is mentioned in the text, but detailed numbers could not be confirmed within the provided range. Preprocessing is costly because the target model must be run twice over the entire training set, with and without the image. Additional fine-tuning is needed to get video performance.

Why it matters

VLM serving costs are rising because of long reasoning responses. This paper shows numerically where text-oriented speculative decoding loses out when carried over unchanged, and pushes lossless speedup further by changing only the training objective. Measurements were taken in SGLang, and the code and models are released.

Key terms

Speculative Decoding
A technique in which a small drafter proposes several tokens ahead and a large target model verifies them all at once. It speeds up generation without changing the output distribution.
Diffusion Drafter
A drafter that uses block diffusion to fill an entire draft block in a single forward pass. With no sequential generation cost, it can use a deeper model.
Acceptance length (τ)
The average number of tokens the target accepts per verification step. The longer it is, the greater the speedup.
KL divergence
A measure of how different two probability distributions are. Here it is used to measure how much the image changes next-token prediction.

Original

Authors: Sicheng Feng, Zigeng Chen, Song Wang, Gongfan Fang, Xinchao Wang · Posted 2026-11-13 · License CC BY 4.0 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.