Paper review · October 27, 2026
RynnUnify: Toward a Unified Embodied Brain
RynnUnify: Towards Unified Embodied Brain
A 16B unified multimodal model combines embodied understanding and video prediction through interleaved reasoning, performing long-horizon visual planning, predictive reasoning and robot control in a single foundation model.
Problem
For a robot to carry out multi-step tasks, it must understand the scene, predict outcomes and turn them into actions. In existing unified multimodal models, scene changes and prediction errors accumulate during long-horizon planning, and static image prediction misses the intermediate dynamics. Benchmarks for evaluating how long-horizon visual planning and video prediction affect reasoning were also lacking.
Method
Following the MoT architecture of Cosmos3, the model connects an 8B autoregressive reasoner and an 8B rectified-flow video generator through joint attention. The reasoner outputs the next subtask, a description of the expected visual change and a length; the generator then produces video, and the last frame becomes the observation for the next reasoning step. Long-horizon generation consistency is improved with a scene memory of four reference frames, a temporal prefix from the end of the previous clip, and self-forcing. There are two extensions: an 8B agent trained with reinforcement learning to decide whether to call video generation, and a World-Action Model built with latent planning and joint video-action training. Rynn-VP and Rynn-GEU were newly created for evaluation.
Results
- The authors report the highest average embodied-understanding score among the unified multimodal models evaluated.
- Its overall Rynn-VP score was 9.75 points higher than Cosmos 3 Nano.
- The agent extension improved the Rynn-GEU average by 6.0 points over direct reasoning.
- The World-Action extension achieved a 51.5% success rate on 20 selected composite tasks in RoboCasa365 and an average of 46.7% on three tasks with a real robot (Astribot S1).
Limitations
Rynn-VP and Rynn-GEU are benchmarks the authors built themselves. Only 20 selected tasks in RoboCasa365 and only three real-robot tasks were evaluated. Comparisons of understanding performance are limited to unified multimodal models. Because a 16B model runs video generation repeatedly, inference cost may be high.
Why it matters
This is a concrete example of the "robot brain" direction that combines understanding, prediction and action in a single model. It shows whether imagining the future through video and making the next plan from that result can carry over to real robot control.
Key terms
- MoT(Mixture-of-Transformers)
- An architecture in which the understanding reasoner and the generation generator each have their own Transformer parameters but exchange information through attention.
- Interleaved video-language reasoning
- An iterative process of planning the next step in language, predicting its result as video, and interpreting the predicted video to make the next plan.
- World-Action Model(WAM)
- A model that jointly learns to predict world changes (video) and robot actions, connecting predictive ability to actual control.
- Self-forcing
- A technique that feeds frames the model generated itself as conditions during training, reducing error accumulation in long generation.
Original
Authors: Minghao Zhu, Bohan Hou, Kun Xiang, Zixuan Wang, Yinzhou Tang, Yifei Yang, Zhikai Wang, Tianyi Zhang, et al. · Posted 2026-10-23 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.