Spatiotemporal Facial Action Unit Detection using Twin Cycle Autoencoders for Driver Monitoring
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
The authors developed a new method called the Twin Cycle Autoencoder (TCA) to better detect subtle facial movements indicating driver tiredness or distraction. Their system combines two types of analysis: one that separates facial features from a person's identity in still images, and another that looks at how these features change over time. This approach helps the model learn from lots of unlabeled driving videos and improves detection of quick or faint facial actions. They tested TCA on several datasets and found it worked better than previous methods, especially in spotting signs of fatigue and yawning. The system also runs fast enough to be used in real cars for driver safety.
Authors
Sai Sidharth D
Abstract
Driver monitoring systems (DMS) increasingly rely on facial cues to infer drowsiness, distraction, and cognitive load in real time. Facial Action Units (AUs), grounded in the Facial Action Coding System (FACS), provide an objective and interpretable representation of such states, but their automatic detection in the driving context is complicated by low and variable illumination, partial occlusion, head-pose variation, and the subtlety and short duration of relevant AU activations. Existing AU detectors largely treat spatial appearance and temporal dynamics separately, limiting their ability to exploit self-supervisory signal from abundant unlabeled driving video. We propose the Twin Cycle Autoencoder (TCA), a spatiotemporal architecture composed of two coupled cycle-consistent autoencoder branches: a Spatial Cycle Autoencoder that disentangles AU-relevant appearance from identity through image-level cycle consistency, and a Temporal Cycle Autoencoder that enforces forward-backward consistency over latent AU trajectories to capture onset-apex-offset dynamics. The two branches are coupled through a cross-branch latent alignment loss and fused via an attention module before multi-label AU classification. We evaluate TCA on the DISFA and BP4D benchmarks and on an in-cabin naturalistic driving dataset, and observe consistent improvements over CNN-RNN, 3D-CNN, and graph-based AU baselines, particularly for low-intensity and rapidly transitioning AUs relevant to fatigue (AU45, AU43) and yawning (AU26). We further show the model sustains real-time throughput on an embedded Jetson Xavier NX platform, supporting its use in production-grade advanced driver assistance systems (ADAS).