Papers for

customer service analytics teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Calliope finds topics in private ai conversations without embeddings

CALLIOPE: Private Insights into AI Use, Without Embeddings

Abstract: We introduce CALLIOPE, a top-down framework for discovering topic hierarchies in large corpora of unstructured data (e.g., LLM conversations), while supporting differential privacy (DP). Unlike traditional and even recent LLM-augmented approaches that ultimately rely on clustering vector embeddings, our framework leverages LLM capabilities directly for the entire pipeline. Experiments demonstrate that this approach provides better utility over existing (embedding-based) methods in the DP setting, especially in the high privacy regime. We evaluate the quality of CALLIOPE outputs using a variety of traditionally studied clustering metrics, as well as some novel utility metrics that we introduce.

Tue 20 OctMachine LearningArtificial IntelligenceComputation and Language
The gist
People often want to discover topics in large amounts of text, like conversations with AI, while keeping the data private. Traditional methods use number-based summaries called embeddings but can leak information when privacy is strict. The authors created CALLIOPE, a new approach that uses AI language understanding directly instead of relying on embeddings. This method offers better results, especially when strong privacy protections are required.
Open → 2610.28797v1

Shared sentiment evidence improves continuous multimodal sentiment prediction

Shared Sentiment Evidence from Frozen Multimodal Models for Continuous Sentiment Prediction

Abstract: Continuous multimodal sentiment analysis estimates polarity and intensity from linguistic, acoustic, and visual cues that may disagree. Hard judgments compress sentiment alternatives, while learned fusion interfaces tie task adaptation to model representations. We introduce Shared Sentiment Evidence (SSE), an output-space interface with fixed view and sentiment-option semantics across frozen backbones. Four queries yield five-option distributions whose probabilities, entropy, and cross-view relations form a 28-dimensional vector for CatBoost regression. On CH-SIMS-v2 and CMU-MOSI, the Qwen-based interface improves on supervised joint-only readout. Matched Ridge and CatBoost controls assess the contributions of additional views, deterministic summaries, and nonlinear readout. With the same interface specification and fixed readout settings, separately fitted heads also improve on joint-only prediction for three additional frozen backbones. These results support learning continuous sentiment from reusable output coordinates.

Wed 14 OctMultimedia
The gist
Predicting emotions from speech, text, and facial cues is tricky because these signals can disagree. The authors propose a method called Shared Sentiment Evidence (SSE) that uses fixed output representations from separate models to better combine different views of sentiment. Their approach improves continuous sentiment prediction on popular datasets without retraining the core models. This suggests it’s possible to learn sentiment continuously using reusable, shared outputs across various pretrained models.
Open → 2610.19981v1

Speech judges overweight loudness and emotion without clear reasoning

Louder, Longer, Livelier: Acoustic Shortcuts and Underspecified Rationales in Speech LLM Judges

Abstract: LLM-as-a-judge is widely used for evaluating text, but extending this paradigm to speech requires models to interpret acoustic as well as linguistic evidence. This introduces a modality-specific risk: a speech judge may treat a perceptually salient cue as evidence of quality even when that cue is irrelevant to the target criterion or receives more weight than human listeners give it. We call this behavior an acoustic shortcut. To study it, we audit six speech LLM judges using controlled manipulations of intensity, content richness, and emotional delivery. We evaluate both pointwise scoring and pairwise comparison, using human preference calibration to interpret the results. The judges consistently reward louder audio, prefer content-rich speech more strongly than human listeners do, and map emotional delivery into quality preferences. These effects are most visible in pairwise comparison, while pointwise scores often obscure them. More concerningly, the accompanying rationales rarely identify the acoustic cue that changes a judgment and instead repeatedly rely on a limited vocabulary, leaving them acoustically underspecified. Together, these findings show that reliable speech judges must both resist acoustic shortcuts and ground their rationales in the acoustic evidence behind their decisions. To support reproducibility and future audits, we also release SpeechJudgeAudit, the controlled stimuli and evaluation tools used in this study.

Tue 29 SeptSound
The gist
Evaluating spoken language with large language models (LLMs) adds challenges because these models must consider sounds as well as words. The authors found that these speech-model judges often give higher scores simply based on louder volume or emotional tone, even when these cues don’t actually indicate better quality. They also found the models’ explanations rarely mention the actual sound clues influencing their decisions, making their reasons unclear. This means speech LLM judges need to better focus on relevant audio features and explain their choices based on those features.
Open → 2609.36979v1