← Key papers

Paper review · November 14, 2026

Duo: A Reciprocal World Model for Real-Time Two-Way Conversation

Duo: A Reciprocal World Model for Full-Duplex Interaction

MultimodalSpeech & audioGenerative media Advanced Upvotes 35 cs.CV inclusionAI

By tying a language model that decides intent and a Diffusion model that generates speech and video onto a single timeline, the authors built real-time two-way video conversation with the user, including interruptions.

Problem

Existing world models focused on simulating the physical world. Users could control the world only with low-degree-of-freedom inputs such as keyboard, mouse and text. There was no intelligence inside the world to perceive and respond to the user in real time.

Method

The authors proposed the Reciprocal World Model (RWM) and built Duo v1 as its implementation. The Intent Module is a multimodal language model that autoregressively outputs <speak>+text or <wait> for every 17-frame (about 0.71-second) chunk. The Realization Module is a causal audio-video Diffusion Transformer initialized from MiniMax-H3 that generates speech and video together, conditioned on Intent's hidden state. All inputs are aligned to a single timeline, and per-module attention masks block access to future information. Finally, Dual rollout Distillation is applied. DMD is performed conditioned on the text generated by Intent and the past audio and video generated by Realization itself, yielding a 4-step streaming generator.

Results

Limitations

This is a technical report. The provided text contains no quantitative evaluation of latency, generation quality or conversational appropriateness, and no baseline comparisons. The authors themselves present v1 as a first step. A large share of the training data is scripted, synthesized video calls.

Why it matters

It shows an architecture that combines interfaces that see, hear and speak in real time, such as AI avatars or virtual agents, into a single system. The design in which the language model directly decides 'speak or wait' is a useful reference for handling natural interruptions and conversational timing.

Key terms

Full-duplex
A communication mode in which both sides can speak and listen at the same time. One side can interrupt or react while the other is still speaking.
Intent / Realization
Intent is the cognitive module that decides when to say what. Realization is the generation module that expresses that intent as speech and video.
DMD(Distribution Matching Distillation)
A technique that distills a Diffusion model that needs many steps into a Student model that generates in fewer steps. It trains by matching output distributions.

Original

Authors: Hengyuan Cao, Biao Gong, Guoshan Lu, Qiyuan Zhang, Lin Liu, Yiding Tian, Yilei Jiang, Yuxin Ma, et al. · Posted 2026-11-11 · License arXiv 비독점 배포 라이선스 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.