Summary
Monitoring people’s movements to detect falls or track activities is important, especially for those who use wheelchairs or have mobility challenges. The current datasets don’t capture enough detail for recognizing actions as they happen. To fix this, the authors created SAFER-Activities, a big collection of videos with detailed labels showing different actions, including those involving wheelchairs. They tested different computer models to recognize these actions and found approaches that work better within the same settings and can also adapt to new situations. The dataset and their code are shared openly to help others improve fall detection and activity tracking.
fall detectionaction recognitionsmart healthcare monitoringwheelchair useframe-level annotationskeleton modelsRGB videodomain shiftdatasetmultimodal fusion
Authors
Diwas Lamsal, Pramod Wickramatilake, Jednipat Moonrinta, Mongkol Ekpanyapong, Matthew N. Dailey
Abstract
Smart healthcare monitoring systems require precise action recognition to ensure well-being and timely intervention in critical situations such as falls, particularly for mobility-challenged individuals. Existing datasets are often clip-based, lacking the frame-level detail needed to recognize actions online, as they unfold. To address this, we introduce SAFER-Activities, a dataset for fall detection and physical activity monitoring, with a dedicated subset for wheelchair use scenarios. It comprises over 66 hours of video data captured by multiple cameras, with 85,310 action instances and frame-level annotations for 30 action classes. We benchmark action recognition on SAFER-Activities with 2D and 3D skeleton models, RGB models with frozen backbones, and multimodal fusion strategies, and evaluate on in-lab, out-of-distribution, and cross-dataset test sets. Skeleton-based models generalize best under domain shift; fusing frozen RGB features with the skeleton stream improves in-domain recognition over the baseline CNN1D, most clearly on the wheelchair subset, but degrades out of distribution. Cross-dataset and qualitative evaluations confirm that models trained on SAFER-Activities transfer well to unseen environments and external fall data. To support research on robust fall detection and activity monitoring, we release the dataset and code at https://safer-activities.github.io/.