Robotic blimps understand human intent from pushes and voice commands
HINT-Blimp: Human INTent Inference from Multimodal Cues for Robotic Blimps
Robotics
Summary
Controlling robots with joysticks or tablets can be slow and distracting because people have to focus on the device instead of the robot. This paper introduces a way for robots, specifically flying blimps, to understand what people want just from a few simple cues like pushes and spoken commands. The robot guesses the person's goal and movement style in real time, improving navigation especially when obstacles are involved. Tests with people showed the robot could quickly figure out the intended direction most of the time after only a few interactions.
What this means in practice
- •For robotics engineers: Build robots that interpret sparse physical and verbal signals to navigate with minimal operator attention.
- •For drone operators: Operate aerial blimps more intuitively by combining gentle nudges and voice inputs to direct movement and avoid obstacles known only to the pilot.
Authors
Subhadeep Koley, Benjamin Greenberg, Yifei Simon Shao, Juan Aceros, Nadia Figueroa, David Saldaña
Abstract
In human-robot interaction, traditional interfaces such as joysticks and handheld tablets introduce latency into navigation tasks and require the operator's explicit attention on the device, instead of the robot. We propose a new human-robot interaction framework in which a human communicates intent directly through sparse multimodal signals such as physical pushes and spoken commands. Human intent is represented as a parameterized linear dynamical system (LDS) that encodes the desired goal and motion behavior. The robot estimates this intent (parameters) online using a particle filter, where each particle represents a candidate LDS hypothesis and is reweighted online as new information becomes available. We validate this framework on a robotic blimp, whose inherent compliance and collision tolerance make it well-suited for repeated physical interaction. Experiments with multiple participants across 300 trials show that combining pushes and voice commands identifies the intended goal in 86% of trials within at most five interactions, with most trials resolved in two. The inferred dynamical systems can also produce curved trajectories that avoid obstacles known only to the human.