Task-Oriented Formation Decision via Reinforcement Learning: Herding an Attacking Swarm
2026-08-10 • Robotics
Robotics
AI summaryⓘ
The authors study how a group of robots can form different shapes to work together on a herding task, where some robots (defenders) try to control others (attackers) who are better at moving and unpredictable. They made a new way to describe the robot formation with a small set of numbers, allowing the shape to change smoothly instead of sticking to fixed patterns. Using reinforcement learning, they trained a system that adjusts these numbers to handle many attacker strategies seen during simulation. Their method worked better than others in tests with many robots and was shown to be practical with real robots.
multi-robot systemsformation controlparameter optimizationreinforcement learningherding taskadversarial strategiesmaneuverabilitysimulationsrobotic platforms
Authors
Zhaozong Wang, Guibin Sun, Jinyong Chen, Rui Zhou
Abstract
Multi-robot systems can accomplish tasks that are difficult for a single robot by organizing into task-specific formations. Different from existing studies on multi-robot shape formation, we here study the task-oriented formation decision problem, with a focus on the herding task. This task is challenging due to the attackers' superior maneuverability and their unknown strategies. To address these challenges, we propose the following novel results. First, we encode the formation shape using a low-dimensional parameter vector. This parametric representation reformulates the formation decision as a parameter optimization problem, thereby resolving the limited flexibility of predefined shapes. By optimizing these formation parameters, the defenders' maneuverability disadvantage is mitigated through a formation shape that continuously adapts to task requirements. Second, we develop a reinforcement learning-based policy to regulate the formation parameters. Trained offline in simulations covering diverse attacking strategies, the learned policy can effectively handle adversarial unpredictability during online deployment. Comparative simulations against three baselines demonstrate that our method can successfully accomplish challenging herding tasks. Additional scalability simulations further verify its applicability to simulated scenarios involving dozens of robots. We also validate the practical feasibility of our method on a physical robotic platform with 3 attackers and 7 defenders.