SeetaPsych offers unified toolkit for facial behavior measurement
SeetaPsych v1.0: An Open-source Computer Vision Toolkit for Behavior-based Psychological Measurement
Computer Vision and Pattern Recognition
Summary
Measuring emotions and other psychological signals from face videos usually requires many separate tools that don't always work well together. The authors created SeetaPsych, an open-source software that combines different computer vision methods to analyze facial expressions, heart rate, and where someone is looking, all in one system. This toolkit also makes it easier to run these analyses at large scale with code or through a simple web interface.
What this means in practice
- •For human computer interaction designers: Integrate multiple facial and gaze analysis tools into one pipeline for designing and evaluating interactive systems sensitive to user emotions and attention.
- •For behavioral health app developers: Build apps that monitor heart rate and emotional signals from face videos without needing separate specialized systems for each measurement.$Commercial implications: Enables creation of integrated mental and physiological monitoring apps with standardized, open-source facial analysis components.
Authors
Jiabei Zeng, Chiqin Li, Kaizhou Li, Fei Chang, Yong Li, Yuanhao Zhao, Dan Han, Wenqiang Yang, Xilin Chen, Shiguang Shan
Abstract
Automated visual analysis opens new avenues for behavior--based psychological measurement. Nevertheless, existing technological modules are typically scattered across task specific systems with heterogeneous interfaces and disparate deployment requirements. In this work, we present SeetaPsych v1.0, an open source, unified and extensible computer vision toolkit designed to extract psychologically relevant signals from facial images and/or face based videos. The current release encompasses four major core modules aiming at behavior--based physiological perception: unified face based emotion analysis (simultaneous facial expression recognition, facial action unit detection, and valence--arousal estimation), camera based heart rate estimation, screen point--of--gaze estimation, and scene gaze following. A suite of auxiliary preprocessing modules for human centric visual analysis is also included, comprising face detection, facial landmark detection, and head detection. These functionalities are encapsulated within a modular Pipeline/Runner architecture that automatically resolves attribute dependencies, constructs computation graphs, and support intermediate result sharing among modules. SeetaPsych provides standardized Python APIs to facilitate reproducible, large scale analyses, alongside an interactive WebUI for rapid, code--free method evaluation. Overall, SeetaPsych offers an integrated and accessible visual measurement platform for research in psychology, behavioral science, human computer interaction, and related fields.