DuplexAct-Bench evaluates full-duplex speech across diverse behaviors
DuplexAct-Bench: Broadening Full-Duplex Speech Evaluation toward Proactive Interaction across Diverse Behavioral Requirements
Computation and LanguageHuman-Computer Interaction
Summary
Communicating naturally in conversations means knowing when to talk, when to listen, and how to respond smoothly. Existing tests for speech systems only check a few speaking behaviors and situations. The authors created DuplexAct-Bench to test speech systems on six types of conversational behaviors in both English and Chinese. Their tests show current systems often struggle to join in conversations appropriately, especially in timing and content. This benchmark helps improve how machines handle real-time talk and interruptions.
What this means in practice
- •For voice assistant developers: Evaluate and improve when and how voice assistants take turns and respond during live conversations using diverse speech behaviors.
- •For call center software teams: Test call automation systems on managing interruptions and proactive speech in multilingual customer interactions to enhance natural flow.
Authors
Keyue Xing, Wentao Ding, Mengmeng Wang, Wenming Tu, Zilong Zheng, Yipeng Kang
Abstract
Existing full-duplex speech benchmarks cover only subsets of real-time interaction behaviors, often under limited contextual conditions. We introduce DuplexAct-Bench, a bilingual benchmark that systematically covers six complementary behaviors, from interruption and yielding to proactive initiation, active silence, and backchanneling, across Pre-session, In-session, and No-explicit conditions. Across 1,290 English and Chinese streaming trials, we evaluate 12 full-duplex speech systems on both Timing and Content. Results reveal substantial variation across behaviors, conditions, and systems, as well as frequent mismatches between semantic quality and behavioral timing. These findings show that current systems remain far from robustly managing when, whether, and how to participate as real-time interaction unfolds. Project page: https://alitaxky.icu/DuplexAct-Bench/