World action models (WAMs): robot models that predict actions and their outcomes together

Why it was needed: the gap in policies that only learn actions

The core of teaching a robot a task is the policy. A policy is a function that looks at camera images and a task instruction and outputs the next joint and gripper commands. Recently, VLAs (vision-language-action models), which put a policy on top of a pretrained vision-language model, have become widely used. However, the Forward Dynamics Models study paper points out a limitation: VLAs learn without explicit supervision about future world states. In other words, they imitate the actions in demonstration data but do not learn how those actions change the world.

On the other side is the world model. A world model predicts "if I move this way in the current scene, what will the next scene look like?" But as the World-Action Model paper paper notes, existing world models are trained only on image prediction, so there is no guarantee that their internal representations capture the structure needed for control. They may draw pictures well without knowing what is relevant to action.

A world action model (WAM) combines the two in one model. The idea is to train action prediction and future observation prediction together, so that actions gain the future as an extra signal and the world model learns structure related to actions.

Key idea: a driver who pictures the result before moving

An experienced driver pictures which way the car will slide before turning the wheel. Steering (the action) and the scenery that will change as a result (the future observation) are bundled together in their head. A WAM turns this intuition into a model architecture. The DroneWAM study paper explains that a WAM lets the model anticipate how candidate actions will change future observations and act based on the predicted outcomes.

The important point is that predicting the future is not a goal in itself. To predict the future correctly, the model has to capture in its internal representation "what is moving, and what does my action change?" The representation built this way then helps action prediction. That is why, in WAM research, "in which representation space should the future be predicted?" keeps coming up as the central question, more than "how nicely can it draw the future?"

The comparison below summarizes where VLAs and WAMs start from and what they use as supervision signals. The Forward Dynamics Models study paper notes that VLAs start from a pretrained vision-language model (VLM), while WAMs start from a video generation model.

How it works: one body, two outputs

Broken into steps, the flow works like this. First, an encoder maps the current camera images (possibly from multiple viewpoints) and the robot state into a representation space. Second, these go into a shared backbone (the body of the model) together with the task instruction. Third, the backbone outputs both a bundle of actions for the next few steps (an action chunk) and the future state after those actions. Fourth, the robot executes the first part of the action chunk, receives a new observation and predicts again. During training, the actual actions and actual future scenes in the demonstration data supervise the two outputs separately.

Approaches branch depending on how the backbone is designed. Some, like Spatially Aware WAM paper, take a large pretrained video diffusion model (a model that generates video by gradually removing noise) and generate actions and future video together. The advantage is that they inherit visual and physical knowledge learned from internet-scale video. Riemann-1.0 paper is built as an autoregressive model (a model that looks at what came before and predicts the next item in turn) that concatenates observations, robot state and actions into a single causal sequence. DroneWAM paper uses a JEPA architecture to predict only the future state in representation space, without drawing future images directly.

Another feature is that the same model can serve two purposes. Riemann-1.0 paper states that a single model acts both as an executable robot policy and as a simulator that shows future scenes when given actions. The World-Action Model paper paper showed an approach that freezes the learned world model and refines the policy inside it with PPO reinforcement learning, in other words, practice in imagination.

How the field developed: what should represent the future?

The World-Action Model paper paper added an inverse dynamics objective to an existing world model, DreamerV2: the task of "looking at the change between two latent states and guessing which action occurred." This makes the representation capture structure related to actions. On 8 manipulation tasks in the CALVIN benchmark, the average success rate of behavior cloning (learning by directly imitating demonstrations) rose from 59.4% to 71.2%. After fine-tuning with PPO inside the model, it reached 92.8% against a baseline of 79.8%, with 8.7 times fewer training steps.

JOPAT paper tackled the weakness of pixel prediction head-on. When predicting pixels, motion gets mixed up with distractors such as lighting and texture. So it predicts latent video, 2D point tracks that include visibility information, and actions together in a single diffusion transformer. Point tracks represent motion explicitly and hold up under occlusion and movement out of frame. The authors report the largest improvements on long tasks involving occlusion, object interaction and off-screen motion.

In August and September, representations and scale broadened. Spatially Aware WAM paper added depth to RGB. It applies a nonlinear transform to depth values to fit the range a frozen VAE tokenizer accepts, preserving pretrained knowledge without 3D-specific fine-tuning. ReWAM paper used controlled comparisons to conclude that neither "representations that reconstruct well" nor "pretrained perceptual features" alone guarantee a good policy. It therefore organizes DINO features into a compressed world state and lets only the gradient of the action loss flow into the representation bottleneck. The policy decides what the representation contains, and the world model learns how that representation changes. Without video generation pretraining, it reached 93.6% on RoboTwin 2.0.

On scale, Riemann-1.0 paper used more than 200,000 hours of interaction data. It trained in stages under the same objective on first-person human video, handheld gripper demonstrations and trajectories from multiple robots, and reported 94.3% on RoboTwin2.0, 99.0% on LIBERO and 62.6% on RoboCasa-365, a benchmark of long composite tasks. The approach has also expanded beyond robot arms. DroneWAM paper applied a WAM to drone visual navigation and introduced "adaptive rollout," which decides how many steps into the future to imagine depending on the scene. It reduced the average prediction depth from 8 to 4.58 while improving trajectory accuracy.

Where it is heading: smaller, faster, only when needed

Studies from October 2026 focus on cost. MiniWAM paper identified a problem: using the high-dimensional features of a pretrained visual backbone directly as future targets makes training expensive. It therefore built compressed targets containing only the transition information that matters for control, using a PRISM encoder that combines inverse dynamics supervision with feature reconstruction. It cut future feature tokens by 65 times while performing better than predicting the original features, and trained up to 8 times faster. The authors report that with 0.25B parameters it is competitive with much larger WAMs.

SmolWAM paper targets edge devices. Using the 2B-parameter Cosmos Policy as the Teacher model, it keeps only 12 of 28 transformer blocks and redesigns the input interface. It succeeded on 95.5% of 400 LIBERO episodes and processed a 16-action chunk in 57 ms with 2.4 GiB of GPU memory. The authors state that it is 11 times faster than the Teacher and uses 3.2 times less memory while still retaining the ability to predict motion in future scenes.

The Forward Dynamics Models study paper reversed the direction. Predicting the future first and then choosing actions based on it slows inference and lets prediction errors spread into actions. So it lets future prediction be conditioned on action features but blocks the reverse. During training, the actual future supervises the policy through the action features, but at inference time future generation is skipped entirely. It reported an average of 99.4% on LIBERO and up to a 13% absolute improvement over VLA and WAM approaches on real-robot tasks. In short, there is a visible trend of future prediction shifting from "a component that must run at inference time" to "a signal that helps training."

Limitations and open issues

First, there is no right answer for choosing a representation. As ReWAM paper showed, high reconstruction quality does not make a policy better. Spatially Aware WAM paper separately analyzed the correlation between the world model's prediction quality and actual rollout success, which reflects the concern that "drawing the future well" and "doing the task well" do not automatically coincide.

Second, cost and error propagation. WAMs based on video diffusion are powerful but heavy. SmolWAM paper pointed out that the strongest models are too computationally expensive to run on edge devices, and the Forward Dynamics Models study paper noted that choosing actions conditioned on a predicted future lets visual prediction errors spread into actions.

Third, security. TAPDreamer paper is an adversarial example study that attaches a fixed patch, built using only public encoders, to the camera input. It worked across tasks and action structures without querying the target model. A single patch covering about 6.5% of the input dropped FastWAM's success rate from 97.7%→0.0% on 40 LIBERO tasks and from 90.86%→0.0% on 50 RoboTwin tasks. With random patches of the same size, success rates stayed at 81.5% and 79.2%. Because many tasks share the same visual encoder, if one encoder is compromised, all of them can be compromised together.

Fourth, how to read the numbers. Most success rates in this article come from simulation benchmarks such as LIBERO, RoboTwin and CALVIN, and on LIBERO many models are already clustered near 99%. Real-world evaluations differ in tasks and conditions from study to study, making direct comparison difficult. Benchmark scores should not be read directly as real-world performance.

Further study

To grasp the concept first, the World-Action Model paper paper is a good start. Its architecture adds a single inverse dynamics objective to a familiar world model (DreamerV2), so it shows most simply "why action signals make representations better." Reading JOPAT paper and ReWAM paper next brings the field's central question into focus: "if not pixels, what should be predicted?"

For the big picture, look at Riemann-1.0 paper and Spatially Aware WAM paper. The former represents the autoregressive approach with large-scale data, and the latter represents reusing pretrained video diffusion models. DroneWAM paper is a good example of applications beyond robot arms and JEPA-style latent prediction.

For the latest trends, reading MiniWAM paper, SmolWAM paper and Forward Dynamics Models paper in that order reveals the progression "compression → lightweighting → skipping the future at inference time." If you are considering real-world deployment, be sure to also read TAPDreamer paper to check robustness issues in advance.