← Key papers

Paper review · November 14, 2026

BirdDreams: A Reactive Driving World Model That Jointly Generates First-Person and BEV Video

BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model

Autonomous drivingGenerative mediaMobility Advanced Upvotes 46 cs.CV NVIDIA

By jointly generating camera video and simplified bird's-eye-view (BEV) video, the authors built a driving world model that closely follows the ego vehicle's path without HD maps and in which surrounding vehicles react plausibly.

Problem

A driving world model needs to simulate surrounding vehicles reacting appropriately when the ego vehicle's behavior changes. Existing models used layouts from driving logs as conditions. As a result, surrounding vehicles only followed their recorded paths and could not react to the ego vehicle's behavior. On the other hand, learning vehicle motion from first-person video alone mixes up appearance and motion and introduces occlusion, so vehicles disappeared or moved unrealistically.

Method

The authors built a two-branch version of the pretrained OmniDreams video Diffusion Transformer. One branch generates first-person video, and the other generates fixed-scale BEV video that draws roads and vehicles as simple shapes. The two branches exchange information through joint attention in every block. Training used Diffusion Forcing with noise levels sampled separately for each stream, followed by fine-tuning with Resampling Forcing. Ego control works by translating and rotating the most recently generated BEV frame to match the input path and feeding the result in as a soft condition. At inference, both videos are generated chunk by chunk from a single camera image.

Results

Limitations

The provided text contains no specific values for the quantitative metrics, so the size of the improvement cannot be verified. Training BEV is rendered from map and 3D object annotations, so training is only possible on datasets with such annotations. There are two evaluation datasets, and the realism of surrounding-vehicle behavior relies on human evaluation.

Why it matters

In autonomous driving simulation, reproducing how surrounding vehicles react in dangerous situations is essential. This work shows a direction of doing reactive simulation with a single video generation model, without external layouts or traffic models.

Key terms

BEV(Bird's-Eye View)
A bird's-eye-view representation that looks down on the scene from above. Because the scale is fixed, distances moved on screen correspond consistently to real distances, and vehicles hiding one another becomes less of a problem.
Driving world model
A model that takes the current observation and the ego vehicle's actions as input and generates future driving video. It can be used as a simulator.
Diffusion Forcing
A training method that assigns a separate noise level to each chunk of a video. It enables chunk-by-chunk autoregressive generation and flexible conditioning.

Original

Authors: Junyoung Seo, Jaewoo Seo, Seungryong Kim, Laura Leal-Taixé, Xiuming Zhang, Seung Wook Kim · Posted 2026-11-10 · License CC BY 4.0 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.