Paper review · November 4, 2026
Wave Forcing: co-designing generation, training and execution for real-time streaming video generation
Wave Forcing: Towards Speed-of-Light Streaming Video Generation
By combining chunk-wise causal attention with wavefront-style pipelined execution, the method achieved long streaming video generation at 134.8 FPS end-to-end on 8 H200 GPUs while keeping multi-step denoising.
Problem
Chunk-wise autoregressive video generation (such as Self Forcing) runs, in sequence, denoising of a chunk and an extra forward pass (store) that updates the K/V history with the clean result. As a result, even with 1-step denoising, each chunk needs two forward passes. In addition, as the Rollout gets longer, image quality degrades and the camera drifts to one side.
Method
Wave Forcing uses chunk-wise causal attention so that a more recent chunk references the less noisy state of an earlier chunk before that chunk finishes denoising. Each denoising step and the store step is assigned its own full model replica, and these run as a staggered pipeline. As a result, once the pipeline is filled, it outputs one chunk per round of parallel forward passes (1F1C). Training has three stages: (1) mixed-noise denoising initialization, (2) motion guidance with Off-policy DMD using Teacher videos and LPIPS, and (3) causal adaptation with On-policy DMD using self-generated history. The execution runtime WaveRT uses Wave Parallelism to send per-layer K/V in advance and overlap communication with computation. It also uses kernel optimization and load-balanced pipelined VAE decoding.
Results
- 5-step WF based on Wan2.1-T2V-1.3B scored 81.64 on VBench Total, on par with 5-step Rolling Forcing (81.15). VBench-Long Quality was also similar at 81.26, compared with RF (81.18).
- Compared with RF, Dynamic Degree rose from 51.39→94.44 on short videos and from 36.63→91.45 on 100-second videos. Subject Preservation was also higher at 97.10 versus RF (92.75). However, Camera Drift was slightly higher at 0.37 versus RF (0.17).
- Causal Forcing had the highest VBench-Long Dynamic Degree at 96.61, but with Subject Preservation of 71.11 and Camera Drift of 11.14, the authors rated its stability as 'unusable'.
- 4-step WF recorded 147.6 FPS for the DiT and 134.8 FPS end-to-end on 8 H200s. This is 7.07× the single-GPU baseline (19.06 FPS).
- In DiT-only tests with SageAttention+W8A8, WP delivered up to 2.53× the throughput of the faster of Ulysses and Ring Context Parallel across six model sizes from 1.3B to 14B.
Limitations
It needs as many full model replicas (GPUs) as there are denoising steps. The logical K/V transfer volume grows with the square of the number of steps. The authors also note that pipeline-filling latency remains for the first output, and that as denoising gets faster, overhead in the output path such as the VAE becomes visible. Generation quality evaluation is limited to the 1.3B model and an 832×480, 16 FPS setting. Camera Drift is somewhat higher than Rolling Forcing.
Why it matters
For real-time interactive video generation and world model serving, throughput and stability over long videos are key. This work shows that generation architecture, training and distributed execution need to be designed together to reduce latency while preserving the quality of multi-step denoising.
Key terms
- 1F1C
- The state in which, once the pipeline is filled, one completed chunk is output per round of parallel forward passes. It contrasts with NF1C (N forward passes per chunk) in sequential execution.
- Chunk-wise causal attention
- A mask that uses full attention within a chunk and, across chunks, lets a chunk reference only earlier (less noisy) chunks. Because it does not depend on later chunks, steps can be executed in an overlapping way.
- DMD(Distribution Matching Distillation)
- A training method that Distills a Student model, which generates in fewer steps, to match the output distribution of a Teacher model.
- Wave Parallelism
- A parallel execution method in which the replica for an earlier step sends per-layer K/V in advance, hiding communication behind computation.
Original
Authors: Yijie Jin, Yuxuan Liu, Jiajun Xu, Ye Huang, Chuzhi Wang, Jiayang Li, Ruiyang Chen, Chenling Hong, et al. · Posted 2026-10-31 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.