Organization feedback summaries often drop rare critical employee voices
Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening
Artificial IntelligenceComputation and LanguageComputers and SocietyHuman-Computer Interaction
Summary
When companies use AI to summarize employee feedback for leaders, some voices get lost. The researchers found that even though criticism tends to be shared more than praise, many unique or less common concerns often disappear in summaries. They created a way to measure how much employee voices are kept or lost and found that summaries favor feedback that is repeated or popular rather than based on sentiment or importance. The work shows that simple checks for positive or negative tone miss bigger issues in which concerns go unheard.
What this means in practice
- •For human resources teams: Audit and improve automated summary tools to make sure less frequent but important employee concerns are not omitted.
- •For tool developers for workplace communication: Design summarization systems that report on voice retention rates and highlight underrepresented feedback themes in reports to managers.
Tested on one dataset.
Authors
Thilo Tamme, Anton Hantel, Bijan Khosrawi-Rad
Abstract
Organizations increasingly route employee feedback to leaders through large language model (LLM) summaries, an unaudited layer that silences already-spoken voice. We introduce a Voice Retention / Representation Ratio metric for representational bias in summarization and apply it to a bilingual (English/German) corpus of 2,586 free-text responses from a global professional service company. First, employees supply criticism more reliably than praise (withholding praise is 82 times more common). Second, across 45 leader-summaries the pipeline filters by popularity, not sentiment: criticism survives, yet a concern voiced once is dropped 86% of the time, with short and German-only content lost on the same axis (theme retention 0.14 vs 0.74; German directional). Controlling for frequency, sentiment has no independent effect; the harm is prevalence-driven, which sentiment-only audits miss. A targeted prompt recovers only named themes. We contribute the metric, field evidence, and a disaggregated voice-retention card.