Paper review · October 27, 2026
Reading, Writing, Sharing and Interacting with the World Through Video: A Survey Toward Unified Video World Models
Reading, Writing, Sharing, and Interacting with the World through Video
A survey that groups video understanding, video generation, unified models and world models into four operations (reading, writing, sharing and interacting) from a "world modeling" perspective, and sets out how research directions have shifted and which challenges remain.
Problem
Video understanding models (Video-LLMs), video generation models, unified understanding-and-generation models and video world models have each developed separately. All of them deal with "the relationship between observed video and the world changing behind it," but there has been little work comparing them within one framework. Existing surveys cover only individual directions such as world models, 3D/4D or long-form video generation.
Method
The survey defines observed video as a partial, lossy, temporally ordered projection of a latent temporal world. On this basis it divides research into four operations: reading (inferring world properties from video), writing (synthesizing video from conditions), sharing (unifying understanding and generation in the same mechanism) and interacting (predicting and updating world state in response to actions and observations). It distinguishes video-level temporal consistency from world-state consistency, and classifies world models along two axes: prediction space (observation space vs. latent state) and transition interface (passive prediction → action-conditioned rollout → closed-loop interaction). Tasks, representations, models, datasets and benchmarks are all reviewed through this framework.
Results
- The authors report that research emphasis is shifting from clip-level recognition and perceptual quality toward persistent state, controllable change, physical and causal plausibility, and evaluation grounded in interaction and downstream decision-making.
- The survey presents a knowledge graph that maps representative work such as Genie, GAIA-1, Cosmos, V-JEPA 2 and Dreamer 4 to the four operations and to finer technical threads (e.g., observation-space rollout, latent-state planning, long-term consistency and memory, real-time responsiveness).
- It identifies long-term consistency, physical and causal grounding, measurable synergy between understanding and generation, and standardized evaluation of world properties as the main open problems.
- A reading list, taxonomy, code and data are released on the project page.
Limitations
As a survey paper, it contains no new experiments or quantitative comparisons. The authors themselves caution that strong performance on individual video tasks does not imply coherent world modeling. The taxonomy is a perspective proposed by the authors, so the boundaries may shift under a different framework. The provided text omits the second half onward, so the contents of later sections could not be checked.
Why it matters
As video generation models and world models for robotics and autonomous driving converge quickly, the survey offers criteria for telling whether a model produces "good-looking video" or models a "coherent world." It can serve as a map for practitioners choosing models or designing evaluation metrics.
Key terms
- Video world model
- A model that learns from video data how an environment changes over time and predicts future states given actions or conditions. It is used for planning, simulation and policy learning.
- Temporal consistency vs. world consistency
- The former means whether video frames connect smoothly; the latter means whether object identity, relationships and physical laws behind the video are maintained. The authors argue the two should be evaluated separately.
- Action-conditioned rollout
- A method that takes the current state and a given action and repeatedly predicts the next state (or video). It is the core interface of world models built as game simulators or driving simulators.
Original
Authors: Shengqiong Wu, Meng Luo, Bobo Li, Ziwei Liu, Shuicheng Yan, Ming-Hsuan Yang, Philip Torr, Hao Fei · Posted 2026-10-22 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.