Mobile model edits video emotion in real time on smartphones
Learned Parametric Emotion Editing: Real-Time Affective Filtering for On-Device Social Media Video
Computer Vision and Pattern RecognitionHuman-Computer Interaction
Summary
Many people spend a lot of time on social media, where content can sometimes be emotionally intense or overwhelming. The authors created a fast method to soften the emotional intensity of videos, making them feel calmer without blocking content or forcing breaks. Their method uses a small neural network to change videos' look in real time, running efficiently on a regular smartphone. This approach lowers how stimulating videos feel, similar to grayscale filters, but keeps better picture quality.
What this means in practice
- •For mobile app developers: Integrate real-time emotional intensity adjustments into social media video apps to promote calmer viewer experiences on smartphones.
- •For content moderation teams: Use affective video filters to reduce emotionally intense content without blocking or coercive interruptions, improving user well-being.
Authors
Musa Rochi, Marcel Schubert, Christoph Gebhardt
Abstract
Problematic internet use affects a growing share of the population, yet common interventions, e.g., time limits, blocking, forced breaks, are coercive and easily circumvented. We explore a less restrictive alternative: adapting the emotional intensity of visual content. Prior work has shown that optimization can steer an image's affective content, but its per-image optimization cost makes it impractical for real-time deployment. We instead learn a model that predicts this transformation in a single forward pass: a MobileNetV4 backbone with FiLM-based emotion conditioning outputs parameters for differentiable global transformations. This replaces prior iterative optimization (80 s per image) with a single 3.7 ms forward pass. In a user study (N = 54), the model reduced viewer-reported arousal relative to unedited images, comparably to the grayscale well-being filter, while being rated higher in perceived quality. We integrate the model into an Android app that adapts Instagram video in real time, sustaining 60 fps on a Samsung Galaxy S23.