Kezek separates two-person mono talks into clear speaker tracks

Kezek: Splitting Mono Two-Person Conversations into Per-Speaker Tracks for Full-Duplex Dialogue Data

Artificial Intelligence

Summary

Many conversations are recorded with both speakers combined into one audio track, which makes studying how people talk to each other harder. The authors present Kezek, a tool that splits a recording of two people talking into separate audio tracks for each speaker, preserving all overlapping speech and quick responses like 'yeah' or 'mm-hm'. Their method works faster and more accurately than existing tools, keeping short replies and avoiding confusing who is speaking. This helps build better dialogue models that understand natural conversation more fully.

What this means in practice

  • For speech technology developers: Create datasets with clean, per-speaker audio from common two-person mono recordings for better spoken dialogue system training.
  • For call center analytics teams: Improve analysis of customer-agent conversations by separating overlapping speech into distinct speaker channels without losing quick responses.

Authors

Bagzhan Karl, Alizhan Karl

Abstract

Full-duplex spoken dialogue models learn turn-taking, overlaps and backchannels from conversations recorded with one channel per speaker, but most available conversational audio is a single mono mix. We present Kezek, a model that turns a mono recording of exactly two people into one track per person with the original length and timing; overlaps and short replies ("yeah", "mm-hm") stay on the track of the person who said them. Kezek decides who speaks when and who is who once for the whole recording (stage 1: a two-voice activity model guided by a speaker fingerprint trained to be insensitive to phase), and a conditioned time-domain separator only carries out that decision (stage 2), with outputs that always sum to the input. Standard separation metrics miss the failures that matter for dialogue data, so we report short-reply retention, leakage onto the other person's solo, channel swaps and coverage. Against the open pipeline pyannote -> DialogueSidon on the same recordings, Kezek keeps more short replies (0.83 vs. 0.69 on Open Yap test; 0.80 vs. 0.71 over the other's speech on unseen telephone speech from CallHome English), mostly because it returns the whole recording while the external pipeline drops part of it (on the audio both return, reply retention is about the same: 0.859 vs. 0.865 on Open Yap); it also leaks less, swaps channels less often, and runs 4-5 times faster (about 230 hours of audio per GPU-hour). We also report negative results and an ideal-mask ceiling that locate the remaining gap: quiet replies under a louder voice. Code and weights are released under permissive licences.