GameVerse: A Minute-Scale Gameplay Dataset for Long-Horizon Interactive World Models

Problem

Interactive world models start to show visual breakdown and lose scene consistency after about one minute. The authors attribute this more to a lack of data than to model capacity. Most existing gameplay data consists of clips under 10 seconds, with only a single-line caption lumping together the whole clip, and reliable public camera trajectories are rare.

Method

The authors collected 11,002 YouTube gameplay videos (4,692 hours, 1080p, 30–60 FPS). They used TransNetV2 (detecting abrupt scene cuts) and CLIP semantic distance (detecting gradual transitions) to cut scene-consistent segments of up to 120 seconds. They then applied a multi-stage filter: OCR-based subtitle and HUD removal, stability and UI checks, deduplication, and temporal quality assessment based on COVER and VBench. Per-frame camera poses were estimated with ViPE and then filtered for sudden jumps, high-frequency jitter and long-term drift. Finally, Kimi-K2.5 was used to add four levels of captions (global static information, local static information, dynamic events and agent actions), aligned to event boundaries rather than fixed time windows.

Results

• The final dataset has 69.7K clips, 1,655 hours and 349.6 million frames, of which 1,358 hours are continuous 2-minute sequences. The authors report that this is about an order of magnitude longer than existing gameplay datasets.
• The authors conclude that among the public datasets compared (Sekai, GameGen-X, Matrix-Game and others), only GameVerse has all of long continuous video, verified camera poses, multi-level captions and interaction.
• The authors report that fine-tuning a strong video generation model on GameVerse improved most control metrics on a 30-second benchmark over a game-domain baseline, including VBench temporal consistency and visual quality and WorldScore content alignment and 3D consistency.

Limitations

Specific figures and comparison models for the fine-tuning experiments could not be confirmed within the provided text. Long-horizon evaluation is based on a 30-second benchmark, so direct verification of improvement in minute-scale rollouts is limited. Poses are ViPE estimates, which differ from engine-level ground-truth poses, and captions were generated automatically by a VLM. Because the source is YouTube video, action (keystroke) labels are not directly provided.

Why it matters

With long-horizon consistency a key bottleneck for world models, it provides large-scale, minute-scale training data that can be released publicly. In a field that has relied on private engine data from large labs, it could become a shared, reproducible foundation.

Key terms

Interactive world model

A generative model that takes action or camera input, generates the next scene and creates a controllable virtual environment.

Camera trajectory verification

The process of filtering non-physical jumps, jitter and accumulated drift out of estimated per-frame camera poses so they can be used as a training signal.

Multi-level structured captions

Captions that describe static scene information, dynamic events and the main character's actions separately. Each element can be given as a separate condition during generation.

Original

• arXiv abstract
• PDF
• HTML full text
• Code
• Project
• Hugging Face

Authors: Kang He, Wenshuo Peng, Chuanhao Li, Zihui Gao, Xiaojie Xu, Zhengyuan Lin, Yuzhe Ding, Donghong Ji, et al. · Posted 2026-10-27 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.