Full duplex models now predict natural backchannel replies in conversation
Controlling Backchannels in Streamable Full-duplex Models
Computation and LanguageHuman-Computer Interaction
Summary
People often say short words like "uh-huh" during conversations to show they are listening, even while the other person is still talking. Most AI models that handle spoken conversation don't plan for these backchannel replies very well. The authors designed a simple addition to existing models that predicts when to produce these backchannels based on the model’s own internal signals. This method works across small and large models and makes the AI sound more like a real person, according to human judges.
What this means in practice
- •For dialogue system developers: Improve conversational AI by making backchannel responses better timed and more frequent based on internal model signals.
- •For human-robot interaction engineers: Create robots or assistants that respond with natural acknowledgements during human speech for more engaging interactions.
Authors
Maike Züfle, Peter Polák, Sefik Emre Eskimez, Jan Niehues, Peter Bell, Ondřej Klejch
Abstract
Backchannels, brief acknowledgements like "uh-huh" produced while the other party may still be talking, are central to natural conversation, but full-duplex spoken dialogue models rarely model them explicitly. We introduce a lightweight backchannel head that predicts, from a full-duplex model's own hidden states, when a backchannel should begin. Once this probability crosses a tunable threshold, a backchannel is force-decoded. Attached to both a 7B (PersonaPlex) and a 1B (F-Actor) model, it generalizes across scale. Probing confirms the hidden states anticipate real human timing, and generation evaluation shows more frequent, better-timed backchannels. Human raters judge the resulting backchannels on par with real ones.