Large language models use debate and graphs to generate better research ideas

Distill, Debate, Discover: Grounding Multi-Agent Research Ideation in Graph-Distilled Insights

Computation and Language

Summary

Generating research ideas is hard and sometimes needs input from many experts. The authors developed a system called InsightAgents that uses groups of AI agents with skills based on summaries of scientific papers connected as graphs. These agents debate and build on each other's ideas to find good research problems and refine how to tackle them. Their system was tested on many topics in language processing and scored better than other idea generation methods.

What this means in practice

  • For ai research teams: Generate and refine novel research proposals by combining AI agents that integrate insights distilled from scientific paper networks.
  • For product innovation teams: Use AI-supported debate frameworks to collaboratively identify and evaluate new technology development challenges and approaches.

Authors

Qianran Ma, Muxin Liu, Grace Kim, Xiao Liang, Chris Callison-Burch

Abstract

Large language models (LLMs) are increasingly used to support research idea generation and scientific discovery. Multi-agent debate offers a collaborative approach to this task by bringing together diverse perspectives to develop research ideas. However, a key challenge is how to equip agents with distinct and meaningful expertise to guide their contributions. Inspired by how researchers draw on accumulated research experience during collaboration, we introduce InsightAgents, a framework that constructs literature-grounded agent expertise by distilling insights from graphs of related papers. Guided by insights that trace evolving research approaches, reveal underlying assumptions, and identify unresolved challenges, within a line of research, agents jointly identify a research problem and propose, challenge, and refine potential approaches through debate to develop a research proposal. We evaluate InsightAgents against three research idea generation frameworks on 100 NLP research topics through pairwise comparisons by two LLM judges. In both online and offline knowledge settings, InsightAgents achieves the highest average score across the five evaluation dimensions under both judges.