Deep learning improves fall prediction in irregular walking patterns
A Multimodal Label Forecasting Method for Aperiodic Visuo-Motor Time Series
Robotics
Summary
Predicting when a humanoid robot might fall is hard because its movements don't follow regular patterns. The authors studied this problem using video and motion data from robots walking in unpredictable ways. They created new test sets and found that existing methods struggle with such irregular movements. To fix this, they designed a new deep learning model that better handles these unpredictable signals and showed it outperforms previous approaches on both simulated and real robot data.
time series forecastingdeep learningtransformer modelsMLP modelsperiodicityegocentric visionproprioceptionhumanoid locomotionmultimodal datai.i.d sampling
Authors
Borui He, Garrett E Katz
Abstract
Deep learning models have been increasingly applied to Time Series Forecasting (TSF) in recent years. Transformer-based and MLP-based models have both been used effectively on many real-world TSF regression benchmarks, and there is ongoing debate as to which family of methods is best. While these benchmarks have drawn much attention, it is also worth noting that many current datasets and methods assume approximate periodicity in the time series. In this work, we focus on a new TSF task without periodicity: anticipating falls during humanoid locomotion, on the basis of egocentric vision and proprioception. When the locomotion trajectories are sufficiently diverse, periodicity is violated. We contribute two new benchmark datasets (one from simulation, one from real hardware), showing that periodicity is violated and recent deep TSF methods struggle on these benchmarks. We also propose a novel deep learning architecture that exploits both endogenous and exogenous variables and a training process that rigorously enforces i.i.d sampling of training examples. Our results show statistically significant improvement over prior art in multiple experimental conditions, by 12.73% or more on the real data and 10.40% or more on the simulation data. Code and datasets will be available upon acceptance.