Skeleton action recognition accuracy improved by reducing entity bias
De-biasing Skeleton-based Action Recognition with Convex Hull Adaptive Shift
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Recognizing human and object actions from skeleton data is harder when the way the data is centered introduces bias. The authors found that the usual way of setting a world coordinate origin causes different parts of skeletons to be seen in biased ways, which confuses recognition. They created a new method called CHASE that adaptively shifts the coordinate origin inside the shape formed by the skeleton points, reducing this bias. This helps existing recognition models work better and is flexible for different types of skeleton data.
What this means in practice
- •For human motion analysts: Improve accuracy of systems that classify human and multi-entity actions by integrating adaptive bias reduction in skeleton data preprocessing.
- •For robotic system developers: Enhance recognition of robot and human-robot interactions by applying a normalization step that adjusts coordinate origins within skeleton data.
Authors
Mengyuan Liu, Yuhang Wen, Yi Zhang, Songtao Wu, Hong Liu, Junsong Yuan, Beichen Ding
Abstract
Skeleton sequences can represent both individual actions and multi-entity interactions, encompassing human bodies, hands, objects, and robots. Existing approaches to recognize skeleton-based actions and interactions usually adopt a late fusion strategy, which expects individuals are independent and identically distributed to train a robust weight-shared entity encoder. However, observed entity bias in various skeletal data violates this assumption, leading to suboptimal optimization of backbone models that might produce wrong recognition results. This bias arises from the world coordinate system's initial configuration, where the choice of origin often creates bias in representation. To this end, we propose a Convex Hull Adaptive Shift based normalization method to reduce Entity bias (CHASE), improving performance across a variety of skeleton-based action and interaction recognition tasks. To adaptively apply plausible shifts to the input skeletons, we formulate a plug-and-play parameterized network that ensures the relocated world origin lies within the skeleton convex hull, which avoids non-convergence by limiting the search space. To further minimize entity bias, we incorporate an auxiliary objective that leverages pair-wise distribution distances to guide network optimization. To support both single- and multi-entity actions, we propose a sub-entity strategy that offers a consistent formulation for both scenarios. Moreover, CHASE demonstrates compatibility with various intra-skeleton modalities, such as bones and velocities, highlighting its adaptability. Essentially, our method works as a normalization approach to reduce entity bias, enabling subsequent classifiers to achieve improved recognition performance across diverse settings. Extensive experiments on 7 datasets verify our approach by seamlessly integrating with various backbones and significantly boosting their performance.