Stranger, Fan, or Peer? A Systematic Study on the Role of Interlocutor in Persona-Based Dialogue Generation
Computation and Language
Summary
The authors studied how dialogue systems that use speaker biographies behave when who gets to see which biography changes during training, inference, and evaluation. They found that it matters most whether the system saw the biographies during training for how it uses persona details, rather than just at inference time. Showing the interlocutor's biography during training helps reduce the system copying too much biographical text from the target speaker. Also, when only the interlocutor sees the target's biography, some target info can leak into the interlocutor's responses, making it easier to identify who’s speaking. The authors highlight the importance of treating training, inference, and evaluation visibility separately to understand dialogue behavior better.
Authors
Daniela Occhipinti, Malvina Nissim, Marco Guerini
Abstract
Persona-based dialogue systems are usually conditioned on speaker biography, but dialogues involve at least two participants, and who has access to whose biography can vary across training, inference, and evaluation. Prior work often neglected these aspects, obscuring mechanisms that only appear when biography visibility is toggled separately across training, inference, and evaluation, a three-stage factorisation that prior work has largely treated as a single factor. We study this factorisation on a dataset of dialogues paired with speaker's biographies, varying whether the target and interlocutor speakers see each other's biographies during training and inference, and using an LLM as a judge to perform author identification. We find that (i) training-time visibility, more than inference-time visibility, determines whether models express persona traits through dialogue or fall back on copying biographical text (a known problem/phenomenon in persona-based generation); (ii) models trained with interlocutor-biography visibility copy less target-biographical text than models trained without it, while changing visibility only at inference time has a less consistent effect; and (iii) under asymmetric disclosure, where only the interlocutor sees the target biography, target content leaks into interlocutor turns more often, and dialogues containing such traces are easier for the judge to identify, especially when interlocutor turns are visible. These results suggest that biography leakage into generated turns is an artefact of how interlocutor visibility is configured across training and inference, and separating the three stages is necessary.