JEPA

How it works

First, part of an image or video is hidden, and only the visible part (the context) is shown to the model. The model does not reconstruct the hidden part pixel by pixel; instead, it predicts what the numerical representation (embedding) summarizing that part would be. Because the correct answer is also a representation summarized the same way, the model focuses on meaning, such as 'what is where', rather than minor details like individual leaves. It is a type of 'self-supervised learning', which learns from the data itself without people labeling the correct answers one by one.

How it has evolved

The starting point was I-JEPA, from the 2023 paper 'Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture'. By having the model look at one region of an image and predict the representations of several other regions, it showed that meaningful image understanding can be learned without hand-designed data transformations. V-JEPA, from the 2024 paper 'Revisiting Feature Prediction for Learning Visual Representations from Video', extended the same idea to video, building a model that understands both motion and appearance purely by predicting features in video, without help from text or existing image models. In 2025, 'V-JEPA 2' learned from large-scale internet video and then added a small amount of robot video, developing into a 'world model' that can even plan to pick up and move objects after seeing a goal image. In 2026, applications to autonomous driving appeared, including Drive-JEPA (pre-trained on driving video while also learning diverse driving trajectories), WA-JEPA (modified to predict future scenes and vehicle motion together), and Flow-JEPA (which treats future prediction as multiple possibilities rather than a single correct answer, making it more robust to noise).

Why it matters

It can learn the structure of the world from vast amounts of video and images that people have not labeled, which reduces the burden of data preparation. Because it does not redraw pixels, it does not waste computation on unnecessary details and more easily gains meaning-focused understanding. In particular, the ability to predict 'what will happen next' in summarized form is drawing attention as a foundation for world models in fields that require planning actions, such as robotics and autonomous driving.

Things to know

JEPA is not a generative model that creates images or video; it is a way of learning representations for understanding and prediction. So rather than producing visible outputs by itself, it is used as the foundation for other tasks (classification, question answering, planning, etc.). Preventing 'representation collapse', in which the model squashes all inputs into the same representation, is an important challenge, and recent research focuses on improving stability in noisy environments and methods for handling an uncertain future.

Example

In research, JEPA is mainly used to pre-train the 'world understanding' part of robots and autonomous driving systems. For example, in the V-JEPA 2 work, the model first learned how the world moves from internet video, and then, with only a small amount of robot video added, a robot arm in a lab it had never seen before was able to plan to pick up and move objects after looking at a goal photo. In autonomous driving as well, researchers are studying approaches that do JEPA-style pre-training on driving video and then add a module on top that decides the route.

Related terms

World model Embedding Transformer VLA Diffusion model Multimodal

References

• Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture (2023)
• Revisiting Feature Prediction for Learning Visual Representations from Video (2024)
• V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025)
• Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving (2026)
• WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving (2026)
• Flow-JEPA: Robust Latent Dynamics for JEPA World Models via Flow Matching (2026)