Multi task deep learning improves face recognition across poses
A Multi-Task Deep Learning Framework for Pose-Invariant Face Recognition and Facial Landmark Localization
Computer Vision and Pattern Recognition
Summary
Face recognition software often struggles when people’s faces are seen from different angles. The authors present a method that teaches a computer to recognize who a person is and locate key points on the face, like the eyes and nose, at the same time. They adapted an existing face image dataset to help the computer learn both tasks together. Their approach combines two neural networks and uses attention mechanisms to improve accuracy under challenging poses. The results show better identification and face landmark detection when faces are viewed from different directions.
What this means in practice
- •For security system engineers: Create face recognition systems that identify people accurately even when faces are captured from different angles by integrating face landmark detection.
- •For mobile app developers: Develop mobile apps that simultaneously detect facial features and identify users, improving performance in varied lighting and pose conditions.
Authors
Ali Ashkeyev, Zukhra Abdiakhmetova
Abstract
Face recognition systems often experience performance degradation under large pose variations due to changes in facial appearance and visibility of discriminative features. This study presents a multi-task deep learning framework that jointly addresses pose-invariant face recognition and facial landmark localization within a unified architecture. To enable simultaneous optimization of identity recognition and facial structure learning, the original Cross-Pose Labeled Faces in the Wild (CPLFW) dataset was adapted into a multi-task learning dataset by associating facial images with identity labels and five-point landmark annotations. The proposed framework consists of two task-specific branches operating on a common facial input: a ResNet50-based branch for identity recognition and an EfficientNetV2B0-based branch with spatial attention for facial landmark localization. The two objectives are jointly optimized using a weighted multi-task loss, allowing the model to learn complementary semantic and spatial facial representations. The proposed framework was evaluated on the prepared CPLFW-based multi-task dataset. Unlike the original CPLFW benchmark, which focuses primarily on identity recognition, the proposed dataset adaptation incorporates five-point facial landmark annotations to enable joint recognition and localization learning. Experimental results demonstrate that the model achieves a validation recognition accuracy of 81.58% and a facial landmark localization mean squared error of 96.38. The results indicate that combining identity classification and landmark regression within a single optimization framework provides an effective approach for learning robust facial representations under cross-pose conditions The proposed method demonstrates the potential of multi-task learning and attention-based feature refinement for unconstrained facial analysis.