Paper review · November 6, 2026
From Pixels to Actions: A Native VLA Policy Without a Vision Encoder
From Pixels to Actions - Towards Native Vision-Language-Action Policies
By feeding pixels directly into the language backbone without a separate vision encoder and letting an Action Expert read representations from multiple layers, the model achieved stronger and more robust robot manipulation than encoder-based VLAs.
Problem
Most VLA (Vision-Language-Action) models first compress images with a pretrained vision encoder such as CLIP or SigLIP and use only its final output for action prediction. But robot manipulation depends on signals such as local geometry, edges, depth and fine changes in state. These signals are easily lost during compression.
Method
NEO-VLA uses a lightweight two-stage convolutional patch embedder (stride 32) to put pixels directly into the backbone's token space. Images and language are processed together in a 40-layer, 2B-parameter encoder-free vision-language backbone. It uses Native RoPE, which separates time, height and width, and modality-specific attention (causal for text, bidirectional within image blocks). A separate Action Expert trained with flow matching reads the KV cache of selected backbone layers to predict continuous action chunks. The authors varied which layers are read (first-28, skip-28, end-28, full-40) to analyze where the visual information needed for actions appears. For pretraining, they used quality-filtered robot data from nine public dataset families: about 2.23 million episodes, or 9,927 hours.
Results
- The authors report that, with the same Action Expert and robot training data, NEO-VLA consistently achieved higher LIBERO success rates than versions based on Qwen3-VL-2B and Gemma-3-4B. The gap was especially large when the backbone was frozen.
- With the backbone frozen, settings that included early layers consistently outperformed settings that read only late layers. After full fine-tuning, reading both early and late layers worked best.
- Swapping early-layer activations between observations increased prediction error far more than altering late layers. The authors interpret this as evidence that action-relevant information peaks before the final representation.
- The authors report strong performance on LIBERO, LIBERO-plus and RoboTwin 2.0, with especially large gains under distribution shifts such as camera perturbations and image noise.
Limitations
As the authors acknowledge, the compared backbones differ in pretraining method and model capacity, so the performance gap cannot be attributed solely to removing the encoder. Evaluation is mainly on simulation benchmarks (the LIBERO family and RoboTwin 2.0), and real-robot experiments and specific success-rate figures could not be confirmed within the provided text. The full-40 setting uses a larger Action Expert, which makes direct comparison with other settings difficult.
Why it matters
It challenges a premise taken for granted in VLA design: that only the vision encoder's output is used. It shows that which layers' visual information is passed to action prediction can determine performance and robustness. The model code, weights and data have also been released.
Key terms
- VLA(Vision-Language-Action)
- A model that takes images and language instructions and directly outputs robot actions.
- Encoder-free (native) multimodal model
- An architecture that feeds pixels straight into the language backbone through a lightweight patch embedder, without a separately pretrained vision encoder.
- Flow matching
- A generative method that learns a velocity field leading from noise to real data. Here it is used to generate continuous chunks of robot actions.
- Action Expert
- A small transformer separate from the backbone. It predicts actions by referring to the KV cache of backbone layers.
Original
Authors: Weichen Fan, Haiwen Diao, Penghao Wu, Siyuan Hu, Ziqi Huang, Tianqi Liu, Mingze Sun, Ziyi Wang, et al. · Posted 2026-11-04 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.