New adaptive method improves low-resolution facial expression recognition
STAR-FER: Adaptive Selective Restoration for Edge-Based Low-Resolution Facial Expression Recognition
Computer Vision and Pattern Recognition
Summary
Facial expression recognition becomes challenging when faces appear small or blurry, especially on devices with limited computing power. The authors propose STAR-FER, a smart way to restore only important parts of a low-quality face image for better emotion detection while saving computing resources. This method predicts which face regions need enhancement and applies focused corrections, improving accuracy without heavy processing. It works well across various levels of image degradation, making it useful for devices like smartphones or edge sensors.
What this means in practice
- •For mobile app developers: Create emotion recognition apps that work accurately on low-quality face images without heavy computing demand.
- •For security system engineers: Enhance facial expression detection in surveillance cameras where faces are often low resolution due to distance.
Authors
ZhongYang Wu, ZhenFeng Gan, Di Zhao, XiaoJun Zhu, HouDe Liu
Abstract
Variable interaction distances continuously degrade facial resolution in edge-based affective interaction, making low-resolution facial expression recognition (FER) difficult under limited computation. Full-face image or feature restoration incurs substantial overhead, whereas lightweight recognition pipelines may miss local cues that remain discriminative after degradation. We propose STAR-FER, a scale-and-task-adaptive selective restoration framework that balances recognition performance and resource constraints. A lightweight image-scale conditioned Gate jointly predicts a pixel-wise spatial ranking and a continuous restoration budget, then applies exact-support projection to obtain an input-dependent free-form mask. The mask-conditioned restorer retains complete facial context while concentrating sparse restoration and residual insertion on the selected support. Across seven degradation scales from x2 to x8 on the complete RAF-DB test set, STAR-FER achieves 80.92% mean accuracy and 71.64% Macro-F1, improving the low-resolution ResNet-18 baseline by 2.25 and 2.71 percentage points, respectively. It surpasses SR-FER by 1.04 accuracy points while requiring 8.84 GFLOPs, a 54.29% reduction, with 28.01% mean mask coverage, and maintains effective recognition at unseen non-integer scales. These results show that spatial-ranking and continuous-budget driven selective restoration reduces spatial modification and computation while maintaining low-resolution FER performance, offering an efficient path to resource-constrained affective interaction.