Sentiment analysis improves with better handling of missing data
Robust Multimodal Sentiment Analysis with Incomplete Modalities via Semantic-aware Completeness based Reconstruction
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Sentiment analysis tries to understand feelings from things like texts, videos, or sounds. Sometimes parts of this information are missing or noisy, which makes it hard to guess the right emotion. The authors designed a way to figure out how much important sentiment information is still there when some parts are missing. They then use this idea to rebuild the missing meaning and improve the overall sentiment prediction. Their method worked well on several test datasets.
What this means in practice
- •For customer service teams: Improve sentiment detection from customer feedback when audio or video inputs are incomplete or noisy.
- •For social media monitoring teams: Enhance analysis of posts combining text and other media despite missing parts to better detect user sentiment.
Authors
Han-Jun Choi, Byunggill Joe, Saim Shin, Jin Yea Jang
Abstract
Recent multimodal sentiment analysis studies increasingly adopt text-centric fusion approaches to exploit the rich sentiment information inherent in the textual modality. However, these approaches often suffer from performance degradation during inference due to partially missing or noisy data in real-world scenarios, especially when sentiment-related cues are missing. To address this issue, we introduce a new completeness estimation approach that quantifies the degree of sentiment-relevant information preserved in incomplete data to guide the reconstruction of missing semantics. Furthermore, we propose a training strategy that stabilizes multi-task learning while jointly optimizing sentiment prediction and completeness estimation. Extensive experiments and in-depth analyses on three benchmark datasets demonstrate that the proposed approach enables more accurate semantic reconstruction, leading to more precise sentiment prediction.