AnyAct improves AI agent actions for better task success
AnyAct: Universal Action for Self-Evolving Agents
Artificial Intelligence
Summary
AI agents often need to use many different tools to complete complex tasks, but choosing the right tools and dealing with changing tool quality can be hard. The authors present AnyAct, a system that helps AI agents pick the best actions from many possible options by filtering relevant tools and removing unreliable ones during use. It also understands feedback from different kinds of information to make better decisions. Tests show AnyAct works better than existing methods and can finish tasks faster with fewer steps.
What this means in practice
- •For software developers: Integrate AnyAct to improve AI agents handling complex workflows involving many tools with dynamic reliability in real-world applications.
- •For automation engineers: Use AnyAct to optimize multi-step automation tasks by efficiently choosing and updating tool actions based on real-time feedback.
Authors
Lingrui Xu, Yangqin Jiang, Jiachang Zhang, Xubin Ren, Chao Huang
Abstract
As large language models (LLMs) advance, AI agents are increasingly deployed in open-world environments to tackle complex sequential tasks (e.g., document processing, cross-application collaboration), relying heavily on actions ranging from GUI operations to semantic APIs. However, three core challenges persist: the "scale dilemma" of massive tool ecosystems exceeding LLM context windows, the "non-stationarity" of tool quality due to updates or outages, and the "heterogeneity" of feedback formats (pixels, text, structured data) creating information silos. To address these, we propose AnyAct, a universal action layer that unifies available capabilities into a self-evolving action space, enabling agents to operate efficiently and reliably in large-scale, dynamic tool ecosystems. AnyAct's core design focuses on two objectives: constructing this action space via hierarchical progressive retrieval (filtering task-relevant actions) and test-time reliability evolution (pruning unreliable actions), and enabling reliability-aware action orchestration through a heterogeneous observation grounding module that unifies multi-modal feedback. Additionally, it defines a hybrid action space (primitive + semantic actions) and optimizes for a balance between task success rate and execution cost. Evaluations on LiveMCPBench and OSMCP (a new benchmark we developed for multi-granularity action collaboration) demonstrate state-of-the-art performance. AnyAct delivers substantial performance gains over baseline methods across various LLM base models on LiveMCPBench and improvements are particularly notable for models with constrained native capabilities. On OSMCP, it achieves 77.27% overall success with only 50 steps, which is half the steps required by most competitors.