DuoMatching: Joint and Marginal Distribution Matching for Few-Step Video Generation

Problem

DMD (Distribution Matching Distillation) compresses multi-step diffusion models into few-step generators, enabling near-real-time streaming video generation. Existing DMD matches only the joint distribution of all frames to a video teacher (joint DMD). This reduces drift, where quality collapses during autoregressive rollout, but per-frame detail and prompt adherence (for example, missing specified feathers) remain lacking.

Method

Training uses a weighted sum of joint distribution matching (video teacher) and marginal distribution matching (image teacher, Qwen-Image). The loss is L_joint + ω·L_marginal. Video VAE latents are compressed along the time axis, so they do not directly match the image teacher's latent space. A lightweight module, LatentBridge, solves this. LatentBridge converts video latents into image latents, conditioned on the previous slice and the frame index, and is pretrained with an L1 reconstruction loss. In addition, Latent Variation Sampling splits the sequence into K segments at points where adjacent latents change most, and samples one frame per segment for supervision. This reduces redundancy from supervision concentrating on static segments. The authors also present a theoretical analysis showing that, for a suitable ω>0, the joint-marginal optimum is closer to the real distribution than the video teacher.

Results

• The authors reported overall preference rates above 80% against every baseline model compared in human evaluation.
• In qualitative comparisons, the authors reported better texture detail, adherence to object counts and attributes (for example, one canoe, a teal cup) and composition than Causal Forcing++, with dynamic motion largely preserved.
• The authors reported that the improvement appeared not only in causal (autoregressive) generation but also in bidirectional generation (based on Wan2.1-T2V-1.3B, with a CausVid baseline). The comparison models were Self Forcing, Reward Forcing, Causal Forcing++ and One-Forcing, and evaluation used VBench and 400 challenging prompts the authors built themselves.

Limitations

The provided text cuts off partway through the experimental results, so detailed VBench numbers could not be checked. The experimental setup is limited to 81 frames, 480×832 resolution, a 1.3B-class generator and a single image teacher (Qwen-Image). The authors themselves say motion dynamics are 'largely' preserved, suggesting marginal matching may trade off against the degree of motion. Training LatentBridge separately requires additional curated video data (29,400 OpenVid clips).

Why it matters

The paper presents a way to raise frame quality and prompt fidelity in real-time video generation while maintaining speed. It could serve as a general framework for reusing the prior knowledge of strong image generation models in video distillation.

Key terms

DMD(Distribution Matching Distillation)

A distillation method that matches the student's generated distribution to the teacher's distribution using the difference between the teacher's and student's scores. It compresses multi-step diffusion models into generators that take one to a few steps.

Joint distribution vs marginal distribution

The joint distribution is the distribution over all frames of a video together, and the marginal distribution is the distribution of a single frame. This paper matches the marginal distribution separately using an image model.

LatentBridge

A lightweight differentiable module that converts temporally compressed video VAE latents into the image VAE latent of a specific frame. It avoids the memory cost of decoding and re-encoding.

Autoregressive rollout drift

A phenomenon in sequential video generation where errors accumulate and the quality of later frames gradually degrades.

Original

• arXiv abstract
• PDF
• HTML full text
• Code
• Project
• Hugging Face

Authors: Jiahao Zhan, Yan Wang, Yongrui Ma, Qunliang Xing, Ruchang Yao, Runtao Liu, Shijie Zhao, Tianfan Xue · Posted 2026-10-02 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.