Clinical knowledge improves AI recognition of physiotherapy exercises
PhysioAI: Clinical Knowledge-Guided Semantic Supervision for Skeleton-Based Physiotherapy Action Recognition
Computer Vision and Pattern Recognition
Summary
Tracking physiotherapy exercises automatically helps people do rehabilitation at home where a therapist can't always be present. Existing AI methods often struggle because rehab exercises are subtle and vary a lot, especially for people with motor impairments. The authors created PhysioAI, which teaches the AI using detailed clinical descriptions of exercises during training, helping it learn better. This approach improves how well the AI can recognize rehabilitation movements just from skeleton data without extra info when actually used.
What this means in practice
- •For physiotherapy device developers: Integrate clinical knowledge-guided AI models to improve automated monitoring of patient exercise performance remotely.$Commercial implications: Enables development of more accurate home physiotherapy tracking products that use AI trained with expert clinical input.
- •For fitness technology companies: Enhance skeleton-based exercise recognition systems to better detect subtle movement differences for personalized coaching and injury prevention.
Authors
Jie Cao, Euijoon Ahn, Anwar Hassan, Jinman Kim
Abstract
Skeleton-based action recognition can support automated tracking of physiotherapy exercises, particularly in remote rehabilitation settings where continuous in-person supervision is impractical. However, most existing methods are developed for large-scale daily-action benchmarks rather than rehabilitation scenarios. Public rehabilitation exercise datasets are typically small, with only subtle kinematic differences between exercise classes. For participants with motor impairments, exercise execution may also deviate from standard movement patterns in amplitude, speed, and coordination, increasing intra-class variability and making reliable recognition more difficult for skeleton-based models. We propose PhysioAI, a clinical knowledge-guided semantic supervision framework that injects structured physiotherapy knowledge into skeleton representation learning. PhysioAI combines graph-based spatiotemporal modelling of human movement with training-time semantic anchors derived from a structured Clinical Knowledge Dictionary (CKD). The CKD descriptions are encoded using a frozen Contrastive Language-Image Pre-training (CLIP) model and projected into an anchor space, where they provide class-specific semantic targets for skeleton representation learning. The resulting CKD-derived anchors are used only during skeleton-model training; inference requires only skeleton inputs. Under subject-disjoint evaluation, PhysioAI achieves $99.03\pm1.34\%$ on KiMoRe Overall, $94.64\pm7.36\%$ on the Hard-67 stress test, and $87.44\pm7.69\%$ on UI-PRMD Overall. These results exceed the strongest comparator for each endpoint by $0.27$, $2.87$, and $1.33$ percentage points (pp), respectively. These findings demonstrate that structured clinical knowledge can serve as an effective source of training-time supervision for physiotherapy action recognition.