AI agents tested on yearlong experiments in simulated greenhouses
SlowLab: Benchmarking Autonomous AI Agents on a Year of Experiments They Cannot Undo
Artificial IntelligenceMultiagent Systems
Summary
AI agents often run quick experiments, but real ones can take months and can't be undone. The authors created SlowLab, a test where AI runs a full year growing crops in simulated greenhouses with hidden conditions. The AI decides when to plant, adjust climate policies, and pick measurements over time. Compared to other methods, the tested AI agents didn’t outperform traditional approaches and couldn’t improve results by reanalyzing past data. This shows challenges in teaching AI to manage slow, complex experiments.
What this means in practice
- •For agricultural technology teams: Evaluate AI methods for long-term crop management decisions under uncertain conditions using SlowLab benchmark.
- •For facility management teams: Test autonomous agents’ ability to run experiments in resource-constrained, slow-changing physical environments similar to greenhouses.
Authors
Ningpei Ding, Zai Wen Feng
Abstract
AI agents are increasingly proposed as autonomous experimenters, yet the benchmarks that test them mostly offer cheap experiments that answer at once. Experiments in physical facilities are different: they take months, occupy scarce equipment and cannot be undone. We present SlowLab, a benchmark in which an agent runs a one-year campaign in four simulated greenhouse compartments on a site whose crop, facility and prices are hidden. The agent decides when to plant which climate policy, whether to stop a crop and which measurements to read while crops grow, and at the end recommends one policy, which a private evaluator scores in weather the agent never saw. In a preregistered study on 48 unseen sites, three low-cost language-model agents ended near a standard grower setting and clearly below a Bayesian-optimisation baseline that starts from that setting. They ran the facility as two synchronous batches, so measurements taken during the season never changed what they planted, and these measurements improved no method's result. Letting other methods re-read the evidence a campaign had collected did not yield significantly better recommendations, which points to where the models start and which experiments they try. Code is released under the MIT license.