Spoken dialogue dataset features interruptions emotions and sound effects
DuplexDrama: A Synthesized Dialogue Dataset with Scenarios, Full-Duplex Behaviors, Expressive Speech, and Sound Events
Computation and Language
Summary
Having realistic conversations with computers is hard because people often talk over each other, show emotions, and react with sounds like laughter. The authors created DuplexDrama, a large set of computer-made dialogues that include these things with different characters and ages. They validated the quality of both the scripts and voices and will share a portion of dialogues in Chinese and English. This helps build better speech systems that understand natural conversation styles.
What this means in practice
- •For conversational ai developers: Train dialogue models to handle interruptions, emotions, and overlapping speech using a richly annotated multilingual dataset.
- •For speech synthesis engineers: Improve synthesized voices with realistic persona-driven emotions and scenario-aware sound effects in training data.
Authors
Qingxiang Guo, Wenke Fan, Shuofeng Zhao, Dawei Yang, Zhiyang Zhou, Yingxin Shang, Hongwei Cai, Zhou Wang, Weixu Wang, Lin Yang, Shuran Zhou, Yang Song
Abstract
We present DuplexDrama, the first synthesized spoken dialogue dataset that simultaneously covers four dimensions: (i) complete persona and scenario settings; (ii) three full-duplex behaviors (interruption, backchannel, incomplete); (iii) expressive speech with persona-aligned emotion labels; and (iv) script-aware sound events. DuplexDrama is built via a 4-stage pipeline; quality validation on both scripts and synthesized audio confirms its quality. We have produced more than 2,000 hours audio data with a 64-voice timbre pool spanning 13 personas and 5 age buckets; 3.8% of all turns carry at least one full-duplex behavior. This data has been validated through internal full-duplex model training. We will release a curated subset of 6,400 bilingual dialogues (800 h, Chinese ~500 h + English ~300 h) to advance full-duplex spoken dialogue model research. Data samples are available at our demo page and LLM-judge evaluation prompts will be released with the dataset.