Papers for

call center analytics teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Kezek separates two-person mono talks into clear speaker tracks

Kezek: Splitting Mono Two-Person Conversations into Per-Speaker Tracks for Full-Duplex Dialogue Data

Abstract: Full-duplex spoken dialogue models learn turn-taking, overlaps and backchannels from conversations recorded with one channel per speaker, but most available conversational audio is a single mono mix. We present Kezek, a model that turns a mono recording of exactly two people into one track per person with the original length and timing; overlaps and short replies ("yeah", "mm-hm") stay on the track of the person who said them. Kezek decides who speaks when and who is who once for the whole recording (stage 1: a two-voice activity model guided by a speaker fingerprint trained to be insensitive to phase), and a conditioned time-domain separator only carries out that decision (stage 2), with outputs that always sum to the input. Standard separation metrics miss the failures that matter for dialogue data, so we report short-reply retention, leakage onto the other person's solo, channel swaps and coverage. Against the open pipeline pyannote -> DialogueSidon on the same recordings, Kezek keeps more short replies (0.83 vs. 0.69 on Open Yap test; 0.80 vs. 0.71 over the other's speech on unseen telephone speech from CallHome English), mostly because it returns the whole recording while the external pipeline drops part of it (on the audio both return, reply retention is about the same: 0.859 vs. 0.865 on Open Yap); it also leaks less, swaps channels less often, and runs 4-5 times faster (about 230 hours of audio per GPU-hour). We also report negative results and an ideal-mask ceiling that locate the remaining gap: quiet replies under a louder voice. Code and weights are released under permissive licences.

Thu 12 NovArtificial Intelligence
The gist
Many conversations are recorded with both speakers combined into one audio track, which makes studying how people talk to each other harder. The authors present Kezek, a tool that splits a recording of two people talking into separate audio tracks for each speaker, preserving all overlapping speech and quick responses like 'yeah' or 'mm-hm'. Their method works faster and more accurately than existing tools, keeping short replies and avoiding confusing who is speaking. This helps build better dialogue models that understand natural conversation more fully.
Open → 2611.16933v1

Speech recognition improves for bilingual online Spanish classrooms

Beyond Aggregate WER: In-domain Adaptation, Learner Speech, Code-Switching, and Speaker Attribution in Online Spanish Learning Classrooms

Abstract: Automatic speech recognition (ASR) increasingly supplies transcripts for educational NLP, but aggregate word error rate (WER) can conceal errors concentrated in learner speech, multilingual turns, and speaker attribution. We compare platform-integrated, open-weight, adapted, and commercial ASR configurations on held-out sessions of online L2 Spanish interaction between L1 English learners and their instructors and tutors. Because English is the learner's shared first language, the interactions contain Spanish-only, English-only, and code-switched turns. Evaluation combines overall WER and character error rate (CER), WER disaggregated by speaker role and language context, and diarization error rate (DER). We also adapt Whisper large-v3 and MOSS-Transcribe-Diarize with LoRA using human-corrected in-domain classroom data. MOSS adaptation produced consistent improvements across speaker roles and language contexts, whereas Whisper showed smaller gains. Domain match proved more important than data volume: Larger out-of-domain bilingual and learner corpora did not outperform the smaller classroom corpus, where learner speech and code-switching are embedded in instructor--learner interaction. The open-weight MOSS adapter was competitive with commercial Muse Voice Transcribe on WER and CER and achieved lower system-provided DER on the recordings eligible for each system. Learner, English-only, and code-switched speech remained more difficult than instructor and Spanish-only speech. The findings demonstrate the value of condition-aware evaluation and interactionally matched data for bilingual educational ASR.

Tue 27 OctComputation and Language
The gist
Transcribing conversations in online Spanish learning classes is tricky because learners switch between Spanish and English and make different kinds of speech mistakes. The authors tested several speech recognition systems and found that adapting models with classroom-specific data helped capture these conversations better than using much larger out-of-domain datasets. Their adapted system identified speech errors more accurately across different speakers and language mixes. This work shows that training with data similar to the target environment is important for good transcription in bilingual educational settings.
Open → 2610.38016v1

Speaker diarization model adapts to 22 Indian languages zero shot

Anuvani-Diarization: Zero Shot Speaker Diarization and Annotation Conventions Across 22 Languages

Abstract: India's 22 scheduled languages are spoken by over a billion people in conversations that mix languages, often within one turn, overlap freely, and take place over telephones, in meetings, and in media. Knowing who spoke when, the task of speaker diarization, is essential to speech applications for such conversation: voice agents conversing with several people at once, speaker attributed transcription of meetings and calls, contact center analytics, and media monitoring. We present a systematic study of speaker diarization across all 22 languages on IndicDiarBench and introduce Anuvani-Diarization, which adapts a self-supervised end-to-end segmentation model to Indian conversation with uncertainty aware silver supervision. Evaluated zero shot alongside nine recent diarization systems, it reaches a duration weighted diarization error rate (DER) of 16.78% on all 1,164 recordings, the lowest of all systems we ran and second only to the best published result (16.0%), and leads the three strongest rivals in 14 of 22 languages. It matches its parent on six international benchmarks. We also show that the benchmark's utterance level annotation convention is a hidden variable worth 1.9 to 4.1 DER points, depending on the system, and that a convention learned from other Indian transcriptions does not transfer.

Wed 14 OctComputation and Language
The gist
Telling who is speaking when in conversations is important for many voice technologies. The authors studied speaker diarization—finding who spoke when—in 22 Indian languages, which are often mixed and overlapping. They created a new model called Anuvani-Diarization that works without prior language-specific training (zero shot) and performs better or as well as recent methods on a large dataset of Indian conversations. They also found that different ways of marking speech segments can affect accuracy scores significantly.
Open → 2610.20327v1

Speech emotion recognition improved by linking audio to editable concepts

From Speech to Editable Concepts: Probing Emotion Recognition with Concept Bottleneck Models

Abstract: Speech emotion recognition (SER) is the task of assigning emotion labels to utterances. Early systems relied on acoustic features, whereas recent approaches combine multiple modalities, most commonly speech and text. Still, performance remains poor on many datasets. Large language models (LLMs) have therefore attracted interest for SER, as they can process diverse inputs jointly with instructions. However, direct audio input raises questions of explainability. To address similar questions in image classification, concept bottleneck models were introduced. This work adapts concept bottlenecks to SER to examine how individual predictions depend on transcripts, acoustic descriptions and speaker attributes. Experiments test three LLMs on CREMA-D, IEMOCAP and MELD, with concepts extracted by separate tools. On scripted corpora, LLMs are strongly biased towards the transcript in the zero-shot setting, which lowers Macro-F1 from 27.8 to 5.8 on CREMA-D. Fine-tuning removes this bias, and the transcript raises Macro-F1 from 41.8 to 45.1. Removing speech rate changes 48% of Neutral predictions to Disgust on CREMA-D; removing intensity level on MELD changes predictions despite little change in Macro-F1. These findings show that aggregate performance changes alone do not capture the effects of concept removal on individual predictions.

Wed 30 SeptSoundArtificial IntelligenceComputation and Language
The gist
Speech emotion recognition is about figuring out the feelings behind spoken words, but it’s still hard to do well. The authors study how large language models make these predictions using words, sound features, and speaker traits combined in clear, editable parts called concepts. They find that focusing too much on transcribed words can make the models worse at recognizing emotions. Their experiments show that when certain sound features are removed, the models’ individual mood guesses change, even if overall scores look similar. This means that just looking at overall accuracy misses important details about how the models think.
Open → 2609.39453v1

Speaker diarization improves by combining neural and embedding methods

Training-Free Affinity Fusion of Neural and Embedding-Based Speaker Diarization

Abstract: Speaker diarization systems based on speaker embeddings and neural diarization exploit complementary forms of speaker information, but their intermediate representations are not directly compatible. We introduce Training-Free Affinity Fusion (TFAF), which integrates the speaker structure inferred by a neural diarizer into an embedding-based diarization system. The neural speaker partition is used to condition local speaker representations, from which we construct a continuous affinity matrix and combine it with the embedding-based acoustic affinity before a single global clustering step. The method requires no additional training, shared embedding space, speaker-label alignment, or hard transfer of the neural diarizer's speaker count. Experiments on AMI and CALLHOME show consistent DER improvements over both constituent systems; on AMI, fusion also improves speaker-attributed transcription. Ablations show that the neural speaker partition accounts for most of the gain, while retaining the continuous embedding-based affinities provides additional benefit over hard partition fusion.

Wed 30 SeptSoundMachine Learning
The gist
Identifying when different people speak in audio recordings can be done by using methods that focus on voice embeddings or neural network-based approaches. These two methods work well but use different kinds of information that don't easily fit together. The authors propose a new way to combine these methods without needing to retrain anything, making the process more effective. Their approach mixes the strengths of both methods and tested on real-world datasets showed it reduces errors in distinguishing speakers and improves the clarity of transcripts.
Open → 2609.39162v1

Cue grounded method improves spoken topic segmentation with zero training

Zero-Shot Cue-Grounded Topic Segmentation of Spoken Documents

Abstract: Topic segmentation structures spoken documents into coherent sections, facilitating navigation and downstream understanding. The appropriate granularity can vary substantially, ranging from broad thematic shifts to fine-grained subtopics. Existing LLM-based segmenters, however, often struggle to adapt to this variation, causing them to either merge distinct subtopics or over-segment coherent themes. To address this, we introduce Cue-Grounded Segmentation (CGS), a training-free framework that operates without any task-specific supervision. CGS first identifies phrases that explicitly signal the start of a new topic and uses their sentence positions as segment boundaries. When such cues are insufficient, it falls back to semantic segmentation, guided by the document structure inferred during cue extraction. Across six benchmarks and six LLM backbones, CGS consistently outperforms existing baselines, remains robust to noisy ASR transcripts, and achieves these gains with low API cost on proprietary models.

Mon 28 SeptComputation and LanguageArtificial Intelligence
The gist
Breaking long spoken documents into topics helps people find and understand information better, but it's hard to do this well when topics vary in size. The authors designed a new way that doesn't require teaching the computer beforehand. Their method looks for special words or phrases that usually signal a new topic, and if those aren't clear, it uses the overall meaning to guess where topics change. This method works better than others, even with imperfect speech-to-text transcripts, and is cheaper to run with big language models.
Open → 2609.34425v1

Speech language models improved for emotion recognition with linear classifier

Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition

Abstract: SpeechLLMs have shown strong potential for emotion recognition, yet they read the predicted emotion off a generative decoder not suited for classification: it can emit labels outside the target set and favors frequent classes. We propose a discriminative adaptation that reads the final prompt token's hidden state through a classification head, producing a label in one forward pass without modifying the backbone. Because this readout starts from the hidden state the model would otherwise decode, it gives a controlled comparison of generative and discriminative inference in an otherwise identical speechLLM. We keep the head a single linear layer, trading little accuracy for interpretability: each emotion becomes one direction in the LLM output token space, revealing associated tokens. On IEMOCAP, across two speechLLM architectures, it improves Macro F1 and removes hallucinations, with largest gains on realistic ASR transcripts. Our analysis reveals that these emotion directions encode indirect associations mirroring biases in web-scale text.

Thu 17 SeptComputation and LanguageArtificial IntelligenceSound
The gist
Detecting emotions in speech is tricky because existing models often guess wrong or use labels not in the expected set. The authors changed how a speech language model reads emotions by adding a simple layer that picks the emotion directly from internal data without changing the main model. This approach reduces mistakes and works better, especially when using text from automatic speech recognition, revealing how subtle language biases connect with emotions. It also helps understand which words the model links to each emotion.
Open → 2609.20081v1