Paper review · November 12, 2026
Few-step streaming video generation with Condition Shifting
Few-Step Streaming Video Generation via Condition Shifting
By flipping the text condition so that a single generator also handles fake score estimation, the method removes the separate fake network, lowering the training cost of few-step streaming video generation and improving instruction following.
Problem
Streaming video generation is expensive because the network must be evaluated multiple times for each new block. DMD (Distribution Matching Distillation), used to build few-step generators, requires a separate fake-score network. This network adds training memory and compute, and it must be kept in step with a generator that keeps changing.
Method
The work extends APEX's Condition Shifting to streaming video. Given the original text condition c, the model generates; given the sign-flipped condition (-c), the same parameters predict the velocity of the generated distribution (the fake estimate). This removes the independent fake network. S2-Avatar's frozen Teacher model, Self-Replay Forcing and perceptual regularization are kept as they are. The generator is trained on a correction target built from the difference between the Teacher's and the fake's clean latent predictions.
Results
- On the StreamAV-Bench Progressive track, VIF (video instruction following) was 3.856, higher than S2-Avatar (3.831). On both tracks, VA (visual aesthetics) was also higher at 0.641898 vs. S2-Avatar's 0.632606.
- In a 70-pair human evaluation, net preference over S2-Avatar was +30.0% for instruction following and +4.3% each for consistency and video quality. S2-Avatar was preferred for motion quality, and audio-visual sync was mostly a tie.
- Under the same training setup, peak memory per GPU was 4.4% lower than DMD and time per iteration was 9.2% shorter.
- AQ, AVAlign and SC dropped slightly on both tracks, and AVSync error rose from 1.319 s to 1.341 s, indicating a trade-off.
Limitations
The main comparison is against S2-Avatar alone, which shares the same training data and backbone. Many of the evaluated metrics show only small gains. Audio quality, sync and subject consistency actually got worse, and VQ on the Interactive track also dropped. The human evaluation covers 70 pairs. The efficiency gains (4.4% memory, 9.2% time) are also modest.
Why it matters
Removing the auxiliary network from distillation training reduces training memory and operational complexity for large video models. It offers a practical alternative for building few-step models for long streaming generation, such as real-time avatars and interactive video.
Key terms
- DMD(Distribution Matching Distillation)
- A distillation technique that trains a few-step generator using the difference between the scores of a frozen Teacher and a fake-score network that estimates the generated distribution.
- Condition Shifting
- A method that changes the input condition (here, the sign of the text embedding) so that one network alternates between two roles: generation and estimating the fake distribution.
- Self-Replay Forcing
- A training approach in which the model re-noises and replays parts of sequences it generated itself. The aim is to reduce error accumulation in autoregressive video generation.
- NFE
- Number of Function Evaluations. The number of times the network is run to produce one sample; fewer means faster.
Original
Authors: Deyuan Liu, Jungang Li, Kaiwen Zheng, Kai Jiang, Jintao Chen, Yaole Wang, Ming Lin, Tao Lin, et al. · Posted 2026-11-10 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.