Progressive approach improves action detection from single time labels
PSEE: Progressive Sensor Event Expansion for Point-Supervised Temporal Action Localization
Computer Vision and Pattern Recognition
Summary
Finding exactly when actions happen in sensor data usually needs detailed start and end times, which is expensive to label. The authors propose a method called Progressive Sensor Event Expansion (PSEE) that only needs one labeled moment per action to guess its full duration. This helps train detectors to find actions without changing how they work later. Tests on several datasets show that PSEE creates better guesses of action times and works well even when labels are sparse or collected from different people.
What this means in practice
- •For wearable device developers: Enable action detection with less detailed annotations to speed up training wearable activity recognition models.
- •For health monitoring teams: Improve detection of patient activities using less costly labeling of sensor data for temporal boundaries.
Authors
Jiaxi Yin, Ge Wang, Han Ding, Fei Wang
Abstract
Temporal action localization (TAL) in wearable sensor streams identifies action classes and temporal boundaries, enabling finer-grained activity understanding than conventional action recognition. However, training typically requires costly start--end annotations for every action instance. To reduce this burden, we study point-supervised TAL, where each instance is labeled with only one timestamp and its class. We propose Progressive Sensor Event Expansion (PSEE), which combines semantic activations, sensor-specific transition evidence, and adaptive temporal ownership to recover point-supervised pseudo segments. These segments supervise standard TAL detectors without modifying their inference procedures. Cross-subject experiments on four inertial-sensing benchmarks demonstrate improved pseudo-boundary quality over adapted point-supervised baselines, compatibility with different TAL detectors, and robustness to point sampling. Code is available at https://github.com/joeeeeyin/PSEE.