Meta reasoning improves decision making for long research tasks
Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents
Artificial Intelligence
Summary
Deciding what to study next in a long research project is tough because it's hard to learn from the many steps involved. The authors present a system called MIRA that breaks this problem into two parts: planning what to research and then doing that research. MIRA helps figure out the best next step by looking back at the progress made so far and predicts how valuable continuing the research will be. This method works better than previous ones in tasks like proving theorems and designing new neural networks, and it can also learn to improve itself over time.
What this means in practice
- •For machine learning engineers: Optimize sequential research or design processes by using meta-reasoning to select effective next steps and allocate compute resources efficiently.
- •For automated theorem proving teams: Improve automated proof strategies by applying a learned meta-reasoner to decide which parts of a complex proof to investigate next.
Authors
Ankur Samanta, Yonathan Efroni, Paul Sajda, Kaveh Hassani, Anirudh Goyal
Abstract
Long-horizon research agents must decide both how to investigate and what to investigate next as evidence accumulates. This is hard to learn because such decisions are sparse in long execution traces, and their consequences may emerge several investigations later. We introduce Meta-reasoning for Iterative Research Agents (MIRA), a hierarchical architecture separating research allocation from execution. An outer-loop meta-reasoner curates context from a persistent research record, then writes a work order for the next investigation or ends the episode. A fresh inner-loop executor carries out each work order, making execution part of the transition between meta-reasoning actions. Without policy training, MIRA improves long-horizon inference and allocates additional compute more effectively in theorem proving and open-ended neural-architecture research. Its decision boundaries also provide natural units for credit assignment. At each boundary, we train a generative critic to forecast expected remaining return from partial states, outperforming token-level alternatives. Cross-environment pretraining improves forecasting and adaptation, yielding a transferable prior for valuing partial progress. We use this prior to initialize MIRA-AC, a generative actor-critic jointly trained to forecast remaining return and choose the next investigation, without a separate critic model. MIRA-AC concentrates policy optimization on meta-reasoning decisions, enabling efficient long-horizon reinforcement learning without directly optimizing the longer execution traces they initiate. Training MIRA-AC on the model's own proxy hill-climbing signals improves gold performance across four autoresearch environments; the actor transfers with cross-environment value initialization. Together, these results show that meta-reasoning can be learned as an explicit policy for directing long-horizon autonomous research.