Paper review · October 22, 2026
Ropedia W0: Scaling a locomotion and manipulation world model across robots with human first-person experience
Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
The authors unified human and robot motion into a single format using camera poses and projected skeletons, and trained on 13,000 hours of data to build a first-person locomotion and manipulation world model usable across multiple robots.
Problem
Existing robot world models focus on fixed-arm and gripper manipulation and handle poorly the viewpoint changes from walking around or finger-level manipulation. Each robot has a different action space, so every new robot requires a dedicated interface or extra adaptation. Public first-person human video data often lacks camera trajectory and hand pose annotations, or has inaccurate ones.
Method
The authors built Xp-Data (about 10,000 hours of human data and about 3,000 hours of robot data) from 14 public human datasets, over 6,000 hours of their own 4-fisheye-camera recordings and 13 robot datasets. The data went through VLM filtering, SLAM-based camera trajectory estimation, hand pose estimation, manual annotation and correction of robot keypoints, and multi-stage quality checks. Locomotion is represented with Plücker-ray camera maps and manipulation with skeleton renderings projected onto the image, putting humans, grippers and humanoids into the same conditioning format. These conditions are added to a Wan2.2-based video diffusion Transformer (5B and A14B MoE) to predict future first-person video. Distillation then made real-time generation possible.
Results
- The authors report top performance, from zero-shot to post-training on the full data, on five cross-embodiment benchmarks (DreamDojo, Ψ0, ω-HOME, Xp-Teleop and AgiBot) and on separate human first-person video.
- Increasing human video pretraining data improved prediction performance on both human and robot videos.
- After Distillation, 1-minute real-time Rollouts and real-time VR interaction were possible.
- The authors demonstrated usefulness in applications such as policy evaluation (ranking), model-based planning and initializing world-action models.
Limitations
Specific per-benchmark figures were not visible in the portion of the paper reviewed, so the size of the performance advantage must be checked in the original tables. Aligning robot skeletons requires substantial manual annotation, which adds cost when extending to new robots. The authors' own benchmark, Xp-Teleop, is small at about 1 hour of data.
Why it matters
Through data-scaling experiments, the paper presents evidence that world models for robots can be grown with abundant human first-person video instead of expensive robot data. A common conditioning interface that works regardless of robot type is practical for humanoid policy evaluation and planning.
Key terms
- World model
- A model that, given the current observation and an action, predicts what the world will look like next. Robots use it to imagine the results of actions in advance.
- Loco-manipulation
- Performing walking or moving motions (locomotion) and handling objects (manipulation) at the same time.
- Plücker-ray map
- A 6-channel image representing the direction and position of the 3D ray each pixel looks along. It tells the video model about camera motion at the pixel level.
- Cross-embodiment
- Using one model in common across agents with different body structures, such as humans, gripper robots and humanoids.
Original
Authors: Tianqi Liu, Hanfei Zhao, Qifeng Liang, Hui En Pang, Jiarui Xu, Sayang Mu, Penghao Qiu, YingSheng Zhang, et al. · Posted 2026-10-19 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.