RSIAgent enables autonomous exploration and self-improvement in new environments
RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments
Artificial IntelligenceComputation and LanguageComputer Vision and Pattern Recognition
Summary
Digital agents, like AI programs that do tasks, often struggle when faced with new environments that they haven't learned about. The authors introduce RSIAgent, a system that helps these agents explore unknown environments on their own, learn from their experiences, and remember important details without needing to retrain. RSIAgent uses multiple cooperating parts to explore both broadly and deeply, finding new patterns and challenges in the environment. This approach helps improve the agent’s knowledge so it can perform better on future tasks in that environment without changing its internal settings.
What this means in practice
- •For ai integrators: Enable AI systems to adapt autonomously to new software or hardware environments without retraining their models.
- •For robotics developers: Allow robots to independently explore and understand unfamiliar physical spaces, improving autonomous operation capabilities.
Authors
Sibo Zhu, Shicheng Fan, Xinyue Wang, Wenyi Wu, Kun Zhou, Biwei Huang
Abstract
Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce \textbf{RSIAgent}, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction. RSIAgent coordinates curriculum, actor, and verifier agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a \textbf{broad-then-deep} exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.