Paper review · November 4, 2026
Video generation meets agent harnesses: agentic video generation, agentic world modeling and recursive self-improvement
Video Generation Meets Agent Harness: Agentic Video Generation, Agentic World Modeling and Recursive Self-Improvement
Taking the agent harness around video generation models as the unit of analysis, the authors organized three directions (video production, world model interaction and recursive self-improvement (RSI)) into a single framework.
Problem
Video generation models alone cannot systematically turn a plan into a coherent multi-shot sequence, keep reference images consistent across scenes, or review results and make partial revisions. Using a world model as an agent's interaction environment also requires observation, control and state management. Existing surveys cover video generation, world models, harnesses and self-improvement separately, and have not tied them together around the harness surrounding the video generator.
Method
The authors define a common agent-harness execution loop and apply it to three areas. (1) Agentic video generation: organized by task function, such as script and shot planning, asset management, shot production, editing and watermarking, with a discussion of what should be separated when evaluating such systems. (2) Agentic world modeling: organized by observation, control, execution and state management interfaces. It covers hybrid designs in which a game engine or an agent-written program maintains the world state and the video model only renders, as well as approaches in which agents and world models evaluate each other. (3) RSI: it distinguishes revising the current output; updating harness state, generator parameters and evaluators; and changing the improvement procedure itself, and lays out the paths by which improvements feed back into subsequent development.
Results
- Taking the harness around the video generator as the unit of analysis, the authors brought production, interaction and model development into a single framework. They also presented a table comparing scope and topics with related surveys and studies.
- The authors presented four paths by which improvements can feed back into the video development process.
- The authors concluded that none of the systems they surveyed yet modifies the procedure for training and validating the generator itself.
- As open problems, they listed autonomous video production, lifelong memory, programmable hybrid control, autonomous model development, provenance and cost tracking, and evaluation that links video quality with long-term improvement.
Limitations
As a survey paper, it has no new experimental results. The authors themselves state that existing video research consists only of components of RSI and is not evidence of a case that completes the full loop. The provided text covers only the introduction and comparison sections, so the detailed taxonomy and evidence in each chapter could not be checked.
Why it matters
Video generation services are moving from single model calls to agent pipelines that include planning, review and revision. This paper provides a common vocabulary for what to distinguish when designing or evaluating such architectures. For example, revising the current output and improving later performance are different things.
Key terms
- Agent harness
- An execution runtime around the model that handles context construction, tool exposure, state maintenance, feedback delivery, and permission and budget management.
- World model
- A model that takes actions as input and predicts the next observation or future scenes. It can be used as an agent's interaction environment.
- Recursive self-improvement (RSI)
- A system improving the very procedure that produces later improvements, rather than just repeating the same improvement procedure.
Original
Authors: Liaoyuan Fan, Yuhao Zheng, Dongxing Mao, Hangyu Ran, Siyuan Hu, Weijia Mao, Rui Zhao, Weijia Wu, et al. · Posted 2026-10-29 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.