Paper review · October 7, 2026
Kandinsky 6.0 Video: A Foundation Model for Synchronized Video and Audio Generation
Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation
The authors built an open model that connects a pretrained video stream and a newly trained audio stream with bidirectional cross-attention to jointly generate 5-second video with lip sync and 44 kHz audio.
Problem
Making video with sound from text is much harder than generating either one alone, because the video and audio must match in meaning, timing and emotion. Commercial systems such as Veo 3.1 and Sora 2 perform well, but their code is not public. Open-source models still lag in clip length, resolution and synchronization accuracy.
Method
The authors extended Kandinsky 5.0's CrossDiT into a two-stream architecture. The video stream and audio stream are connected in every block by bidirectional cross-attention (V2A and A2V). There are two models: Lite (3B) and Pro (29B). Pro consists of 19B for video, 5B for audio and 5B for cross-attention. Training proceeded in this order: ① train the audio stream from scratch on 40 million audio tracks ② train both streams jointly on 7 million paired video-audio segments ③ combine domain-specific SFT results with a model soup ④ OmniNFT-based reinforcement learning ⑤ two-stage distillation with π-Flow and adversarial refinement, reducing function evaluations to 10. Output is upscaled to Full-HD by a separate super-resolution model in the K-VAE latent space.
Results
- The authors report that in human side-by-side comparisons, the Pro model was clearly better than the previous version, Kandinsky 5.0 Video Pro.
- The authors report higher scores than the open-source LTX 2.5 on most VABench metrics. Specific numbers are not within the scope of the provided text.
- The authors claim the model is competitive with commercial audio-video generation models, with particular strength in speech quality.
- It supports two modes, T2AV and I2AV. Code, checkpoints and diffusers integration are released under the MIT license.
Limitations
Generation is limited to 5-second clips. The default output is SD resolution, and Full-HD requires a separate super-resolution model. The provided text has almost no quantitative evaluation numbers, and the main comparisons rely on human side-by-side evaluation, which makes objective numerical comparison difficult. The data includes a Russian cultural dataset and captions are mainly in Russian and English, so performance in other languages has not been verified.
Why it matters
Among models that jointly generate video and audio with lip sync, this release makes a large 29B-class model available under the MIT license. Training the audio stream separately first and then attaching it to an existing video model is a practical approach that teams wanting to add sound to an existing video model can reference.
Key terms
- Dual-stream CrossDiT
- An architecture that connects two diffusion transformers, one for video and one for audio, with cross-attention. The two modalities reference each other to stay synchronized.
- Bidirectional cross-attention
- Attention in which video tokens attend to audio (V2A) and audio tokens attend to video (A2V). It serves to keep the two aligned in time and meaning.
- T2AV / I2AV
- A mode that generates video and audio together from text (T2AV), and a mode that generates video and audio conditioned on a starting image (I2AV).
- Distillation (knowledge distillation)
- A technique that compresses a model needing many sampling steps into a model that produces similar results in fewer steps. It speeds up generation.
Original
Authors: Team Kandinsky, Julia Agafonova, Bulat Akhmatov, Mikhail Aksyutin, Grigorii Alekseenko, Anastasia Aliaskina, Olga Androsova, Vladimir Arkhipkin, et al. · Posted 2026-10-04 · License CC BY 4.0 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.