Runtime coordination improves AI scientists choosing work in projects
Can AI Scientists Coordinate at Runtime?
Multiagent SystemsArtificial Intelligence
Summary
AI scientists often work together by following fixed plans made before starting. This paper asks whether AI scientists can instead adjust how they work together while in the middle of a project, just like human scientists do. The authors created a system called Runtime Agent Coordination (RAC) that lets AI scientists pick partners and assign tasks during the project, checking the results without slowing down the process. They tested this system on several AI scientist setups and found that letting AI choose partners at runtime generally helps, though adding extra task rules and checks can sometimes reduce performance. Their work shows the benefits and limits of coordinating AI scientists dynamically.
What this means in practice
- •For ai development teams: Manage AI scientist agents dynamically during complex research tasks to improve collaboration and adapt to new conditions.
- •For software orchestration engineers: Develop runtime scheduling systems that assign and verify work among AI modules without blocking progress.
Authors
Zijian Liu, Yangzhixin Luo, Junyu Lu, Yi Li, Yu Chen, David Xu, William F. Shen, Xinchi Qiu, Xisen Wang
Abstract
Multi-agent AI scientists have shown improving performance across a diverse range of tasks. Yet a common approach is design-time agentic orchestration, which typically relies on fixed workflows. In contrast, human scientists coordinate and adjust their division of labor at runtime. We therefore ask: can AI scientists also coordinate at runtime? To this end, we introduce Runtime Agent Coordination (RAC), which selects agents from existing AI-scientist hosts during execution, assigns scoped work contracts, and provides artifact-grounded verification. Verification informs subsequent agents without blocking transitions or discarding artifacts. We conduct a single-seed exploratory evaluation across Agent Laboratory, EvoScientist, and ARK on ResearchClawBench, preserving host models, tools, and permissions under host-calibrated budgets. Four cumulative conditions separate native execution, runtime communication, runtime selection, and the combined addition of contracts and verification. Runtime selection yields the highest observed mean score for each host; adding contracts and verification reduces these means, with host-dependent outcomes relative to native execution. These results motivate runtime coordination while exposing the limits of additional coordination mechanisms under constrained budgets. Code is available at https://github.com/systemind-team/Runtime-AI-Scientist.