Tool measures body language and speech in video and audio recordings
NICE Toolbox: Nonverbal Interpersonal Communication Exploration Toolbox
Computer Vision and Pattern Recognition
Summary
People communicate using gestures, facial expressions, tone of voice, and more, but studying these signals is usually slow and needs experts to label them. The authors created the NICE Toolbox, a free software package that automatically detects these nonverbal and verbal cues from videos and audio. It combines several advanced methods into one tool that works with multiple cameras and shows results in both 2D and 3D views. This makes it easier and faster for researchers to analyze how people interact without doing all the manual coding.
What this means in practice
- •For psychological research teams: Automate measurement of interpersonal communication cues across modalities to accelerate behavioral studies.
- •For human-computer interaction developers: Incorporate multi-modal nonverbal signal detection into interactive systems to enhance responsiveness to user behaviors.
Authors
Aleksandr Evgrashin, Carolin Schmitt, Timo Lübbing, Ashutosh Jha, Buket Naz Zeren, Sophie K. Bauer, Johannes Kopf-Beck, Anton K. G. Marx, Anamaria Semm, Gökce Ergün, Senya Polikovsky
Abstract
Understanding human communication is a long-standing goal in psychological research. People express themselves through many verbal and nonverbal channels: posture, gesture, gaze, facial expression, and speech. Measurement and annotation of these signals has traditionally required time-consuming manual work by trained human labellers. The Nonverbal Interpersonal Communication Exploration (NICE) Toolbox is a free, open-source Python framework that automatically measures these signals by combining existing state-of-the-art computer vision and speech-processing methods into a single software package. It includes support for multi-camera systems, interchangeable detection algorithms, 2D and 3D visualization of detector outputs, evaluation on the user's custom data, and export to third-party software. With the release of this tool, we aim to make data exploration of interpersonal communication signals scalable, reproducible, and accessible for psychological research.