Paper review · October 7, 2026
In-Distribution Forcing: In-Distribution KV Management for Long Video Generation at Test Time
In-Distribution Forcing for Long Video Generation at Test Time
A training-free technique that matches both how the KV cache is built and how it is read to the configuration used during training. It extends a 5-second model to minute-long video generation while reducing color and motion drift.
Problem
Autoregressive video diffusion models generate short videos well, but beyond the training horizon they suffer from drift: colors and textures shift and motion diminishes. Existing methods only controlled which KV entries to read (KV conditioning). The authors point out that these methods assume the cached KV is in-distribution. Beyond the training horizon, the context referenced when creating the KV (its provenance) is itself a configuration never seen during training, so the KV becomes out-of-distribution (OOD). The authors define this as the KV-provenance problem.
Method
Level 1 (KV caching) uses self-caching. Beyond the training horizon, the first ℓ chunks of the window are cached by attending only to themselves, without referencing earlier KV. The rest are re-cached autoregressively on top of them, so that every KV is produced from a configuration seen during training. Level 2 (KV conditioning) always places the first chunk κ0 in slot 0. The reason is that the causal 3D-VAE encodes the first frame as a single frame, so it cannot be replaced from a distributional standpoint. The most recent L−1 entries are kept in an exact rolling window that re-rotates RoPE as it rolls. No additional training is needed. It was applied to Self-Forcing and LongLive, both based on Wan2.1-T2V-1.3B.
Results
- In 120-second generation with Self-Forcing, the Color Drift score was 72.37, higher than the highest-scoring baseline, Deep Forcing (59.65). Motion Drift was 89.06, higher than ∞-RoPE (85.16).
- At 120 seconds with Self-Forcing, Dynamic Degree was 65.65, higher than MemRoPE (62.90) and the original Self-Forcing (30.04). The authors reported the highest Dynamic Degree in every setting.
- In 240-second generation with LongLive, Color Drift was 71.76, higher than MemRoPE (64.23). Aesthetic Quality was 60.44, higher than the original LongLive (58.94).
- The authors reported that a 2AFC user study also confirmed the drift-mitigation effect.
Limitations
Motion Smoothness is slightly lower than the base model in some settings. For example, at 240 seconds with LongLive it was 98.50, lower than the original LongLive (98.74). With LongLive, Motion Drift was 96.09, slightly lower than MemRoPE (96.88). Experiments are limited to two models based on the Wan2.1 1.3B architecture, 128 MovieGen prompts and 480×832 resolution. Some of the drift metrics were defined by the authors.
Why it matters
It reframes drift in long video generation as a question not only of 'what is read' but also of 'in what context the KV was created.' Because it can be applied directly to existing short-horizon models without retraining, it is practical for applications that need long generation, such as world models and streaming video.
Key terms
- KV cache
- Memory in which a transformer stores the key and value vectors of previously generated content and refers to them through attention during the next generation step.
- Drift (drifting)
- A quality degradation in which errors accumulate as autoregressive generation gets longer, so colors and textures shift and motion diminishes.
- KV-provenance problem
- A problem in which the context referenced when a KV entry was created is a configuration never seen in training, so the KV itself becomes out-of-distribution.
- Self-caching
- A method in which a chunk is cached by attending only to itself, without referencing earlier KV. Because it has no source context, it stays in-distribution even when other entries drop out of the window.
Original
Authors: Jeongwoo Shin, Youngyoon Choi, Sangwoo Jo, Hyunmog Kim, Sungjoon Choi, Joonseok Lee, Jaewoong Choi, Jaemoo Choi · Posted 2026-10-02 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.