LLM agent performance drops under controlled noisy interactions
AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations
Artificial Intelligence
Summary
Computers using large language models (LLMs) perform complicated tasks but can get confused when their inputs have small changes or mistakes. The authors studied how these AI agents react to different kinds of disturbances during interaction and found that their performance often gets worse. They discovered that agents usually do not notice when inputs are changed and tend to keep using bad information, which wastes time and computing power. Their work suggests training methods that teach agents to check their information and handle uncertainty better to make them more reliable.
What this means in practice
- •For ai system developers: Improve training methods to create LLM agents that detect input errors and recover from corrupted interactions.
- •For software testing teams: Use the benchmark pipeline to systematically evaluate robustness of AI agents under realistic noisy conditions.
Authors
Yuanjian Xu, Jinyi Han, Jianing Hao, Yuxuan Sun, Fanyang Lu, Richeng Xuan, Guang Zhang, Zhichao Hu
Abstract
Real-world tasks are often complex and noisy, posing substantial challenges for LLM agents. Yet existing evaluations lack a systematic framework for jointly characterizing potential perturbation targets and types. We use this two-dimensional view to conduct a unified study that quantifies the robustness of current frontier models, traces the mechanisms behind perturbation-induced failures, and draws implications for robustness-oriented training. Through controlled evaluations on APEX-Agents and Toolathlon, we find that perturbations consistently degrade performance, while the dominant vulnerabilities vary across task environments. Trajectory-level analysis reveals that agents rarely recognize altered inputs and often propagate corrupted information without verification. A focused cost analysis further shows that successful perturbed trajectories can require substantially more computation. These findings motivate training objectives that cover diverse forms of noise and reward verification, uncertainty awareness, and recovery. We provide a rigorous benchmark-construction pipeline. Code and data will be released at https://github.com/xuyj233/AgentPerturb.