Multilingual Sentiment Aware Text Summarization A Reinforcement Learning Approach for Consistency Maintenance
2026-06-08 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how reinforcement learning from human feedback (RLHF), which helps language models create better summaries, affects the emotional tone in those summaries. They found that RLHF tends to make summaries sound more neutral, losing the original feelings expressed in the source text, especially when a certain regularization is stronger. The researchers identified that this regularization causes the emotional tones to be suppressed and proposed a way to adjust it so that sentiment can be better preserved without losing summary quality. Their work shows that while current methods improve accuracy and safety, they might unintentionally reduce emotional expression in generated text.
Reinforcement Learning from Human Feedback (RLHF)Text SummarizationSentiment DriftKL RegularizationPolicy AttributionAlignment ObjectivesAffective PropertiesEmotional ExpressivenessLanguage Models
Authors
Mikhail Krasitskii, Alexander Gelbukh, Olga Kolesnikova, Grigori Sidorov
Abstract
Reinforcement Learning from Human Feedback (RLHF) has significantly improved the quality and fluency of large language models in text summarization. However, its impact on affective properties remains insufficiently understood. In this work, we study sentiment drift, a systematic shift toward neutral sentiment in RLHF-based summarization outputs compared to source texts. We conduct extensive experiments across multiple datasets, model architectures, and eight languages to analyze how alignment objectives influence sentiment preservation. Our results show that sentiment drift is a consistent phenomenon that becomes stronger with increased KL regularization strength, indicating a trade-off between alignment stability and affective fidelity. To explain this behavior, we introduce a Policy Attribution framework that decomposes the RLHF objective and quantifies the contribution of its components. Our analysis reveals that KL regularization is the primary driver of sentiment suppression across all settings. Based on these findings, we propose a sentiment-aware modification of the KL regularization term, which selectively reduces constraints on sentiment-bearing tokens. Empirical results demonstrate that this approach mitigates sentiment drift while maintaining summarization quality. Overall, our findings highlight a fundamental limitation of current alignment methods: while they improve factual consistency and safety, they may unintentionally suppress emotional expressiveness. This motivates the development of alignment strategies that explicitly account for affective preservation.