Sparse behavior patterns improve explanation of multimodal depression assessments
From Sparse Representations to Behavioral Insights for Multimodal Depression Assessment
Computation and Language
Summary
Depression can be hard to measure by looking at behavior because many tools mix lots of detailed information in ways that are tough to understand. The authors created BehavDep, a method that breaks down different behaviors into simpler, understandable pieces linked to meaningful patterns. It also handles cases where some data is unclear or just weakly connected to overall depression signs. Their tests show this approach not only assesses depression well but also explains which behaviors and data types matter most.
What this means in practice
- •For mental health app developers: Integrate interpretable models to provide clearer feedback on depressive behavior from multimedia data.$Commercial implications: Enables apps to offer more transparent depression assessments using sparse behavioral signals, improving user trust and engagement.
- •For clinical data analysts: Use sparse factor models to uncover meaningful behavioral contributors in patient video assessments under limited annotation quality.
Authors
Guimin Hu, Zihao Song, Jiachen Luo, Jiayuan Xie, Ruichu Cai
Abstract
Multimodal depression assessment offers a promising approach to analyzing behavioral patterns associated with depression. However, existing methods often rely on dense and opaque multimodal representations, making it difficult to interpret the behavioral patterns underlying their predictions. In this work, we introduce BehavDep, a sparse factor-based framework that decomposes multimodal behavioral representations into sparse latent factors and associates them with behaviorally meaningful concepts through a semantic bridge. To address the mismatch between user-level annotations and heterogeneous video-level behaviors, BehavDep further learns video-level depression tendency scores under weak supervision and aggregates information across multiple observations for user-level assessment. Extensive experiments demonstrate that BehavDep achieves the best overall assessment performance while revealing complementary modality contributions, heterogeneous behavioral patterns across observations, and prediction responses to concept-level editing. These results show that BehavDep provides a structured and interpretable approach to analyzing multimodal behavioral representations for depression assessment.