Paper review · October 14, 2026
GroundingPI: A Grounding Foundation Model for Physical AI Based on Visual Primitives
GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
The authors built a 4B grounding model that generates point and box coordinates as tokens. It achieved the highest average score across 34 benchmarks and also improved robot manipulation and autonomous driving performance.
Problem
VLA and world action models take their perception abilities from general-purpose VLMs or video generation backbones. These backbones struggle to pinpoint 'which object is where' in complex scenes or with small objects. As a result, scarce action data ends up having to carry perception as well.
Method
The model combines a MoonViT-V2 vision encoder with a Qwen3-4B decoder. Points and boxes are converted into coordinate tokens quantized to 0–999 and generated within the same vocabulary as labels, which unifies detection, referring, pointing, OCR, GUI and layout grounding into a single format. Training proceeded in order through multimodal and spatial pretraining, SFT, and GRPO reinforcement learning. The GRPO reward mixes IoU-based coverage with F1, count, and penalties for duplicate boxes, among other terms. A data engine that merges and verifies labels from multiple teacher models was also used. Downstream, the model serves as the visual backbone of an Action DiT, or generates driving trajectories as coordinate tokens.
Results
- It averaged 73.68% across 34 grounding benchmarks spanning 11 perception abilities, higher than the larger GPT-6 Astra (71.54%) among 44 baselines.
- It scored 74.53 on the dense-scene benchmark Dense200 (F1mIoU). GPT-6 Astra scored 65.04 and Qwen3-VL-4B scored 14.02.
- In all four out-of-distribution (OOD) settings of RoboTwin 2.0, it outperformed every backbone compared, with up to a 24.8% (relative) improvement over the strongest backbone.
- On RoboCasa-GR1, using only 50% of the demonstration data, it outperformed baselines trained on 75%.
- Used as the visual backbone on nuScenes, it achieved an average open-loop L2 error of 0.296 m. The authors reported that downstream performance rises as pretraining scale increases.
Limitations
The authors acknowledge that small-object detection, as in VisDrone, remains difficult. Their own model is averaged over 10 runs, while locally run baselines are averaged over 3 runs due to cost, so measurement conditions differ. Some GUI evaluations are marked as having prompt and parser uncertainty. The autonomous driving evaluation is open-loop only, using a single front camera, and the robot experiments rely mainly on simulation benchmarks.
Why it matters
When choosing a backbone for robotics or autonomous driving models, this provides evidence that 'precise perception' is a capability worth developing on its own. At 4B parameters, it may also be usable for latency-sensitive real-time control.
Key terms
- Visual Grounding
- The task of locating, with a box or point, where a target described in text appears in an image.
- VLA(Vision-Language-Action)
- A model that takes images and language instructions as input and outputs robot actions.
- Quantized coordinate tokens
- A method that divides coordinates into an integer range such as 0–999 and represents them as tokens. This lets a language model generate coordinates the same way it generates text.
- GRPO
- A reinforcement learning method that generates several responses to one input and updates the policy by comparing relative rewards within the group.
Original
Authors: Qize Yu, Lianrui Fan, Boyu Chen, Jiaqi Liang, Xini Ding, Yue Chen, Zetian Song, Yuran Wang, et al. · Posted 2026-09-30 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.