Paper review · October 15, 2026
Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes
The authors used reinforcement learning to train a VLM agent that explores city-scale 3D scenes with zoom-in and oblique-view tools, so it can find objects referred to in natural language without a list of candidate objects.
Problem
Existing methods for city-scale 3D grounding rank object candidates extracted in advance. They therefore depend on candidate quality and are slow in large outdoor scenes with many candidates. Indoor methods downsample point clouds and so lose fine detail on small objects.
Method
The scene is represented as an interactive multi-view environment with a global BEV (bird's-eye view) image and view-to-world coordinate transforms. The VLM agent gathers evidence by calling landmark search, high-resolution zoom-in and oblique-view rendering tools, then submits a box in BEV coordinates, which is projected into world coordinates. It is trained on CityRefer with GRPO alone (no demonstration data). The reward consists of an IoU term, a continuous reward based on center distance, a step cost and tool-level shaping rewards that reflect, for example, whether a zoom-in moves closer to the target.
Results
- The authors report the highest Hit@1 and IoU among compared methods on CityRefer
- They report that grounding works without ground-truth candidate boxes and that inference was much faster than the strongest city-scale ranking baseline
- They report improved generalization robustness to other scenes in zero-shot evaluation on STPLS3D-Refer
Limitations
The provided text was cut off just before the experimental results, so specific numbers could not be checked. Training used only CityRefer, and the cross-scene evaluation set (STPLS3D-Refer) was built by the authors themselves. Landmark search depends on place-name information, and because the final box is defined on the BEV plane, precision in the height direction may be limited.
Why it matters
It turns large-scale 3D space exploration from static retrieval into a sequential tool-use problem. The design is a useful reference for applications that locate objects in wide areas, such as aerial inspection or urban digital twins.
Key terms
- 3D visual grounding
- The task of finding the object that a natural-language description refers to within a 3D scene and outputting its location (a box).
- BEV(Bird's-Eye View)
- A bird's-eye image looking down on the scene from above. Here it serves as the base view on which the agent specifies coordinates.
- Reward shaping
- A technique that gives small rewards for intermediate actions (for example, whether the agent zoomed in toward the target) in addition to the final outcome reward, to help the agent learn to explore.
Original
Authors: Yu Wang, Rui Dai, Wei Chen, Yi Wang, Bo Dang, Kaikui Liu, Xiangxiang Chu, Yansheng Li · Posted 2026-10-13 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.