Paper review · November 14, 2026
BirdDreams: A Reactive Driving World Model That Jointly Generates First-Person and BEV Video
BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model
By jointly generating camera video and simplified bird's-eye-view (BEV) video, the authors built a driving world model that closely follows the ego vehicle's path without HD maps and in which surrounding vehicles react plausibly.
Problem
A driving world model needs to simulate surrounding vehicles reacting appropriately when the ego vehicle's behavior changes. Existing models used layouts from driving logs as conditions. As a result, surrounding vehicles only followed their recorded paths and could not react to the ego vehicle's behavior. On the other hand, learning vehicle motion from first-person video alone mixes up appearance and motion and introduces occlusion, so vehicles disappeared or moved unrealistically.
Method
The authors built a two-branch version of the pretrained OmniDreams video Diffusion Transformer. One branch generates first-person video, and the other generates fixed-scale BEV video that draws roads and vehicles as simple shapes. The two branches exchange information through joint attention in every block. Training used Diffusion Forcing with noise levels sampled separately for each stream, followed by fine-tuning with Resampling Forcing. Ego control works by translating and rotating the most recently generated BEV frame to match the input path and feeding the result in as a soft condition. At inference, both videos are generated chunk by chunk from a single camera image.
Results
- The authors reported that on PhysicalAI-AV and nuScenes, BirdDreams follows the input ego path more faithfully than existing driving world models.
- They reported that even without an ego-action condition, it generated more plausible ego motion than existing models.
- In human evaluation under the same ego maneuver conditions, it was rated as having more realistic surrounding-vehicle motion and behavior than the OmniDreams baseline, which uses first-person video only.
- The authors presented cases where surrounding vehicles react when the ego vehicle leaves its recorded path. For example, when the ego vehicle cuts into a lane, the vehicle behind slows down.
Limitations
The provided text contains no specific values for the quantitative metrics, so the size of the improvement cannot be verified. Training BEV is rendered from map and 3D object annotations, so training is only possible on datasets with such annotations. There are two evaluation datasets, and the realism of surrounding-vehicle behavior relies on human evaluation.
Why it matters
In autonomous driving simulation, reproducing how surrounding vehicles react in dangerous situations is essential. This work shows a direction of doing reactive simulation with a single video generation model, without external layouts or traffic models.
Key terms
- BEV(Bird's-Eye View)
- A bird's-eye-view representation that looks down on the scene from above. Because the scale is fixed, distances moved on screen correspond consistently to real distances, and vehicles hiding one another becomes less of a problem.
- Driving world model
- A model that takes the current observation and the ego vehicle's actions as input and generates future driving video. It can be used as a simulator.
- Diffusion Forcing
- A training method that assigns a separate noise level to each chunk of a video. It enables chunk-by-chunk autoregressive generation and flexible conditioning.
Original
Authors: Junyoung Seo, Jaewoo Seo, Seungryong Kim, Laura Leal-Taixé, Xiuming Zhang, Seung Wook Kim · Posted 2026-11-10 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.