VFlash: Fast and Accurate Visual Plan Generation with Flash Primitive Decoding

Problem

VLMs for robotics and autonomous driving generate visual plans such as points, bounding boxes and trajectories autoregressively, one coordinate token at a time. Longer, more detailed plans need more sequential decoding steps, which raises latency. Within a given latency budget, this limits how detailed plans can be and how often they can be updated.

Method

The authors propose Flash Primitive Decoding (FPD), built on DFlash's lightweight block diffusion drafter. The block layout is set in advance from the primitive type and the length declared through a tool call, and the whole primitive is predicted at once as a single block. The drafter is conditioned on features from the frozen target VLM. It is trained on ground-truth coordinate labels, not on the target model's outputs. At inference, per-position vocabulary constraints are applied, and structurally valid primitives are accepted directly without token-matching verification. Low-confidence boxes are regenerated with AR decoding (fallback). Ordinary text still uses standard speculative decoding verification.

Results

• On 3D robot trajectories (InternA1, AgiBot, MolmoBot), all three training variants improved ADE, FDE, SR@5 and SR@10 over AR decoding. The authors report that ADE fell by 18–49% with 3D-only targets and by 28–61% with 2D+3D targets. AgiBot SR@5 rose from 50.3% to 73.1% (3D-only).
• On 2D trajectories, using the same 4B target model, DFD was 0.194 on ShareRobot-Bench (AR 0.202) and 0.126 on VABench-V (AR 0.130). All six error metrics were lower.
• For bounding boxes with AR fallback, mean F1 was similar to AR: 47.79 on COCO (AR 48.68) and 61.78 on Dense200 (AR 60.10). Boxes per second were 2.6–2.8x higher (COCO 8.76 vs 3.15).
• For 3D point motion prediction, the authors report about 25x faster generation, based on a preliminary single-rollout measurement. Robot manipulation pipeline latency dropped by about 80% on Jetson Thor and DGX Spark.
• A world action model conditioned on VFlash plans reached a 70.2% average success rate across four MolmoSpaces task suites (baseline 62.6%).

Limitations

The method does not preserve the AR model's output distribution, so quality is judged only by spatial accuracy. Accepting boxes directly without fallback lowers mean F1 on Dense200 by about 20 points, and F1 at IoU 0.95 is lower than AR. Improvements in dense 3D motion prediction were not consistent: WorldTrack FDE rose by 15% and PWT fell by 7–12%. The 25x speedup is from a preliminary single-rollout measurement.

Why it matters

Robot VLMs can make more detailed visual plans without adding sequential decoding steps. This lets them update plans often while staying within latency budgets on edge devices. The work also shows that AR decoding does not set the quality ceiling for visual outputs.

Key terms

Speculative decoding

An acceleration technique in which a small draft model proposes several tokens ahead and a large target model checks them all in one pass. Usually, tokens are accepted only up to the point where they match the target model.

Visual primitive

A structured spatial output, such as a bounding box or a 2D/3D trajectory, whose coordinate layout is fixed once its type and length are set.

Block Diffusion drafter

A lightweight draft model that uses bidirectional attention to fill several masked positions at once in a single forward pass.

AR fallback

A safeguard that regenerates part of the output autoregressively when the parallel prediction for that part fails a confidence check.

Original

• arXiv abstract
• PDF
• HTML full text
• Project
• Hugging Face

Authors: Yu-Cheng Chou, Shihao Wang, Jin Huang, Chi-Pin Huang, Chiao-An Yang, Xianzhe Fan, Jing Wang, Zhiqi Li, et al. · Posted 2026-11-05 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.