LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search
2026-08-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors focus on a task where a system needs to find people in videos based on both how they look and their unusual actions. They identify problems in existing methods that mix up appearance and movement features, making it hard to spot specific behaviors. To fix this, they design a new model called LightAIR that uses text clues and special math techniques to separate appearance from actions clearly. Their approach also prevents the model from taking shortcuts that ignore important action details. Experiments show LightAIR works better than previous methods on benchmark datasets.
Text-based Person SearchPerson Anomaly SearchAction InversionOrthogonal Null-Space ProjectionRiemannian GradientContrastive LearningAppearance FeaturesAction FeaturesShortcut LearningSurveillance Video Analysis
Authors
Yulun Zhang, Zixu Li, Zhiwei Chen, Zhiheng Fu, Wenbo Wang, Zihang Qiu, Zhilin Wang, Ruxin Wang, Yupeng Hu
Abstract
Traditional Text-based Person Search (TPS) is typically limited to matching static appearance attributes, severely neglecting dynamic action information. The Text-based Person Anomaly Search (TPAS) task bridges this gap, requiring models to locate micro-level specific abnormal behaviors while matching macro-level appearance of pedestrians. However, current TPAS methods face fundamental limitations: external explicit pose estimators are fragile in unconstrained surveillance scenarios, and implicit learning encounters visual decoupling failure under pixel-level entanglement, causing dominant appearance information to easily swallow and contaminate subtle action features. Furthermore, performing contrastive optimization on hard negative samples (``same appearance, different actions'') in conventional Euclidean spaces induces severe shortcut learning. To address these, we propose the Lightweight Action Inversion and Riemannian rectification network (LightAIR). First, it introduces textual semantic priors as anchors via a lightweight action inversion operator to extract pure action features, thereby overcoming visual-inherent coupling. Subsequently, it employs orthogonal null-space projection to constrain appearance features within the orthogonal complement space of action features, guaranteeing strict forward decoupling. Finally, we designed a gradient rectification module that computes the Riemannian gradient to constrain the backpropagation trajectory, forcing the gradient flow to update strictly along the tangent space that preserves decoupling properties, thereby cutting off harmful shortcuts. Extensive experiments on the widely used TPAS and TIPR datasets demonstrate that LightAIR significantly outperforms existing state-of-the-art methods. Codes are available at https://github.com/rainy-london/LightAIR