Paper review · October 31, 2026
IDM: Few-step image generation distillation that adds moment matching on the generator's own features
Intrinsic Distribution Matching
Adding a loss that matches the mean and standard deviation of the generator's EMA features to the DMD correction gave the highest long-text rendering accuracy among the compared methods at 2 NFE.
Problem
Text-to-image generation in one or a few steps leaves little room for refinement, so accurately drawing prompt details (especially long text) is difficult. DMD relies on learned score estimates, and estimation error and finite training can leave distribution statistics not fully matched.
Method
The authors decomposed the reverse-KL gradient field into a Gaussian moment component determined by the mean and covariance and a remaining non-Gaussian residual, identifying statistics that can be explicitly constrained. Building on this, IDM keeps the DMD correction computed with the generator's internal velocity branch as is (no separate fake-score network) and adds a loss that matches the mean and per-coordinate standard deviation of generated and reference samples, using mean-pooled last-layer hidden representations from an EMA copy of the generator. No external feature encoder or discriminator is used. Experiments covered full-parameter training and LoRA on Z-Image-Turbo 6B and Qwen-Image 20B.
Results
- The authors reported that IDM based on Z-Image-Turbo at 2 NFE scored 94.25% on Chinese and 90.20% on English in LongText-Bench, the highest among compared methods at the same compute budget.
- Under the same conditions, it was 2.63 percentage points higher than Z-Image-Turbo++ on Chinese and 0.32 percentage points higher on English.
- In controlled ablations, the authors reported that adding the moment loss improves performance.
- Across the five reported metrics, the generator's EMA features performed similarly to an external DINOv2 encoder.
Limitations
As the authors note, moments (mean and variance) alone cannot determine the full distribution, so the DMD correction must be used alongside them, and the moment loss is an auxiliary objective inspired by the theoretical decomposition rather than a direct estimate of it. It matches only the diagonal standard deviation and does not handle the full covariance. The key metrics focus on a long-text rendering benchmark, and the improvement in English is small (0.32 percentage points). The experimental backbones are also limited to two families (6B and 20B).
Why it matters
Because it matches distribution statistics using only the generator's own features, without an external encoder, it could be a practical way to improve quality when distilling large image models into few-step generators without the burden of an additional model.
Key terms
- DMD(Distribution Matching Distillation)
- A distillation technique that uses the difference between the score of the reference distribution provided by the teacher and the score of the generated distribution to match a fast generator's output distribution to the reference distribution.
- NFE(Number of Function Evaluations)
- The number of times the network is run to produce one image. The smaller it is, the faster inference is.
- EMA (exponential moving average) copy
- A copy of the model maintained as an exponential moving average of the parameters during training. IDM uses this copy as a feature extractor.
- Moment matching
- A training approach that makes statistics such as the mean and variance of two distributions equal.
Original
Authors: Deyuan Liu, Yansen Han, Kaiwen Zheng, Jun Xie, Jiacheng Liu, Zhenglin Cheng, Peng Sun, Jintao Zhang, et al. · Posted 2026-10-29 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.