Assistive systems predict human goals and future actions early
Neuro-Symbolic Hierarchical Intention Anticipation in Human Behavior
Artificial IntelligenceComputer Vision and Pattern RecognitionHuman-Computer InteractionMachine LearningNeural and Evolutionary Computing
Summary
When machines help people, they need to guess what a person wants before the action is finished. The authors made a method that looks at part of what a person is doing and predicts what they will do next and what their overall goal is. Their system combines neural network abilities with logical rules to keep the guesses sensible and organized into levels, like actions, activities, and intentions. It works better than other methods, especially for longer guesses and for new combinations of behaviors it hasn't seen before.
What this means in practice
- •For assistive robotics teams: Improve robot helpers to predict and respond to human goals earlier and more accurately for safer and more helpful interaction.
- •For smart home technology developers: Enable smart systems to anticipate occupant intentions from partial behavior data, allowing proactive and context-aware assistance.
Authors
Farnaz Soleimani, Abdelghani Chibani, Yacine Amirat, Ghazaleh Khodabandelou
Abstract
Assistive autonomous systems must anticipate human goals before an observed behavior is complete. This article formulates anticipation as goal inference from a partially observed multimodal episode together with structured prediction of the remaining behavior, rather than exact motor forecasting. A compact Hierarchical Planning Decoder (HPD) is attached to a frozen neuro-symbolic recognition encoder and predicts, at four ontological levels, the next actions, the remaining activities and low-level intentions, and the episode high-level intention(HLI). The decoder is trained with soft neuro-symbolic regularization combining transition-coherence and hierarchical continuity losses, and is decoded with hard reachability masks that enforce ontological validity at inference. On a compositional four-level benchmark of 15,002 multimodal episodes built over NTU RGB+D 120 features, three headline properties are observed together. The advantage over the strongest sequential baseline grows with the anticipation horizon, from +1.7 points at step 1 to +7.3 points at step 3 (top-5). Under compositional generalization, where one parent association per multi-parent low level intention is held out, this advantage widens to +4.9 points at step 1. At the episode level, 96.8% of anticipated trajectories satisfy the joint logic constraints, above the 88.1% strongest-baseline value and the 73.9% ground-truth floor; soft logic terms alone account for a 59.8 to 71.1% relative reduction of HLI-reachability violations, and the hard masks then eliminate them entirely. Neural generation supplies predictive ranking, symbolic constraints supply onto logical validity, and their combination yields coherent hierarchical anticipation while exposing remaining challenges in compositional goal generalization and unordered set prediction.