ParaAgent improves tool use efficiency with coordinated parallel actions

ParaAgent: Reinforcing Parallel Acting in Open-World Tool Environments

Artificial IntelligenceComputation and Language

Summary

Using AI agents in environments where tools can do many different things is hard because they must try out new capabilities while using known ones efficiently. The authors show that the solution is to organize exploring and working in a coordinated way that happens both in phases and at the action level. They created ParaAgent, which uses multiple actions in parallel and learns through reinforcement with carefully structured rewards. On simulated tests with many tool interfaces, ParaAgent outperformed other methods including GPT-4-based ones, especially in tasks requiring multiple tools.

What this means in practice

  • For software engineers: Build AI assistants that efficiently explore and use multiple digital tools simultaneously in complex workflows.
  • For automation platform developers: Develop smarter automation systems that coordinate parallel tool actions for higher success rates in multi-tool tasks.

Authors

Shengbin Yue, Hongru Wang, Siyuan Wang, Xiaoxin Chen, Wei Chen, Zhongyu Wei

Abstract

Language model agents are increasingly deployed in open-world tool environments, which require balancing exploring unknown capabilities and exploiting known ones. Existing methods face a performance-efficiency tradeoff: they either rigidly decouple exploration and execution or interleave them without coordination. We argue that the key lies not in whether to decouple or interleave them, but in how to coordinate them across granularities. We introduce ParaAct, a structured parallel-action loop that combines phase-level Exploration $\rightleftharpoons$ Execution with action-level parallelism. To learn this loop, ParaAgent combines multi-agent cold-start demonstrations with reinforcement learning under multi-level advantage decoupling, making planning structure explicit and supervising it with step-, phase-, and trajectory-level rewards. Learning is supported by our ToolEnv, a scalable simulator grounded in 50,011 realistic tool interfaces. On two open-world tool benchmarks, ParaAgent-4B achieves the best average success among all baselines, including GPT-4.1 systems, with the largest gains on multi-tool tasks. Behavioral analyses show that these gains stem from this action organization, highlighting its importance for capable and efficient open-world agents.