Paper review · October 9, 2026
Long-WAM: Extending the Context of World-Action Models
Long-WAM: Scaling the Context of World-Action Models
The authors adapt a video model pretrained autoregressively (AR) for robot control so that it can use a longer visual history, and optimize it to run in real time on edge devices.
Problem
Fast robot control needs enough visual history to track object motion and task progress. But the more history a model processes, the slower it outputs actions. In addition, existing WAMs (World-Action Models) mostly repurpose video generators pretrained bidirectionally for causal control. The fact that such models can 'access' a long history does not guarantee that they actually 'use' it.
Method
(1) LongLive2.0-Robot: Starting from the AR checkpoint of LongLive-2.0, the model is further trained on about 10,000 hours (window-equivalent) of robot and first-person video without action labels. It learns to predict the future from the past with Teacher Forcing. (2) Causal-to-causal adaptation: An action expert is attached while the causal structure of the video expert is kept intact. The model first partially denoises the future video latent (stopping at σ=0.9), then generates an action chunk conditioned on that KV cache (IDM-style). (3) Deployment optimization: The authors apply asynchronous execution, streaming VAE encoding, NVFP4 (W4A4) quantization, CUDA Graph, KV reuse within an execution call, and per-device kernel tuning. With these, the model runs in real time on RTX 5090, DGX Spark and Jetson AGX Thor without dropping the future-prediction step.
Results
- RoboCasa GR-1: Increasing the history length from 0 to 19.2 seconds raised the success rate from 63.3% to 78.7%. In contrast, a model initialized from bidirectional pretraining (Wan2.2) showed no net gain. The gap between the robot-domain AR model and the bidirectional model widened from 3.3 percentage points at 0 seconds of history to 17.1 points at 19.2 seconds.
- On LIBERO-Long, a 2.4-second history raised the success rate from 94.5% to 99.5%. It scored an average of 94.4% on RoboTwin 2.0 and 34.9% on DOMINO, the highest among the compared methods.
- On an RTX 5090, generating one action chunk, including future video latent prediction, took 107.4 ms. This is 3.3 times faster than BF16 eager execution.
- It achieved a 95% success rate on dynamic cup stacking with a Unitree G1. On the same task, π0.5 and Fast-WAM did not succeed in any of 20 attempts. When RoboTwin 2.0 was run asynchronously, Fast-WAM's performance dropped by 15.4 percentage points and LingBot-VA's by 45.5 points, while Long-WAM nearly maintained its synchronous success rate.
- On RoboCasa365, combining it with a GPT-6 Astra planner raised the success rate from 31.4% to 54.4%. Using the planner alone gave 25.2%.
Limitations
A separate model was trained and evaluated for each history length. The optimal history length also differed by benchmark (19.2 seconds for GR-1, 2.4 seconds for LIBERO-Long). The authors note that prefill, attention and cache costs grow as the history gets longer. Real-robot experiments used about 20 attempts per task, so the sample is small. Reproduction is expensive because it requires large-scale video pretraining and tuning for specific NVIDIA hardware.
Why it matters
The paper shows experimentally that whether 'longer context' helps a robot policy depends on how the underlying video model was pretrained. It also demonstrates at the system level that a WAM that includes future prediction can run in real time on edge devices.
Key terms
- WAM(World-Action Model)
- A model that combines a world model, which predicts future video, with action generation, so that it produces robot actions based on the predicted future.
- Autoregressive (AR) video pretraining
- A training method in which the model predicts the next frame by looking only at past frames. Unlike bidirectional training, it learns the causal structure that runs from history to the future.
- Asynchronous execution
- A method in which the robot infers the next action chunk in advance while it is still executing the current one. The key question is whether actions connect smoothly where the two chunks overlap.
- NVFP4
- NVIDIA's 4-bit floating-point format. In this paper, both weights and activations are quantized to 4 bits to speed up the video expert's computation.
Original
Authors: Wei Huang, Bohan Zhang, Chenzhi Liu, Isabella Liu, Shuai Yang, Weian Mao, Luozhou Wang, Yicheng Xiao, et al. · Posted 2026-10-07 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.