Robotic systems explore and adapt to find and manipulate objects
From Passive Execution to Active Exploration: Agentic Embodied Manipulation in Realistic Environments
Robotics
Summary
Robots usually follow fixed instructions to pick up or move things, which can be hard when objects are hidden or there are distractions. This paper shows a way for robots to actively explore their environment, learn about objects they need to find, and then figure out how to handle them. The authors built a system with modules for planning, seeing, and moving that work together so the robot can change its actions based on what it discovers. Their approach helps robots complete tasks even when they can’t see everything from the start.
What this means in practice
- •For warehouse automation teams: Build robotic systems that can discover and handle packages not initially visible or clearly labeled.
- •For industrial robotics engineers: Implement robots that adjust their manipulation strategies in cluttered, realistic factory environments with changing object visibility.
Authors
Shilin Ma, Chubin Zhang, Xulong Bai, Zifeng Gao, Shiyi Zhang, Yansong Tang
Abstract
Recent advances in agentic systems have substantially enhanced the long-horizon capability of embodied manipulation. However, many existing frameworks still follow a passive execution paradigm, which limits their applicability to real-world scenarios involving textual semantic cues, distractors, and initially invisible targets. To bridge this gap, we propose an agent-based active exploration framework that enables robots to dynamically interact with the environment rather than merely execute predefined instructions. Specifically, our framework consists of three collaborative modules: a planning module for high-level task reasoning, a perception module for visual scene understanding, and an execution module for low-level manipulation. This design allows the robot to actively acquire task-relevant information, adapt its behavior based on environmental feedback, and complete manipulation tasks under partial observability. Furthermore, we introduce a fine-grained perception-execution interleaving strategy, which tightly couples visual feedback with skill execution to improve exploration robustness. We evaluate our method on a realistic Find-and-Place task, demonstrating its effectiveness in challenging environments where target objects must be actively discovered before manipulation.