Scientific agents adapt their collaboration style to diverse researcher personas
PersonaScienceBench: Evaluating Scientific Agents with Diverse Simulated Researchers
Artificial Intelligence
Summary
Most AI tools for science work with a fixed type of partner, ignoring that real researchers have different ways of working and preferences. The authors created PersonaScienceBench, which uses simulated researchers with various personas to test how well scientific AI agents adapt their communication and problem solving. They found that agents show different strengths depending on the simulated researcher they interact with, even if final answers score similarly. This helps understand how AI can better collaborate with diverse human researchers.
What this means in practice
- •For scientific software developers: Create AI assistants that customize their interactions based on different scientist user profiles to improve collaboration outcomes.
- •For ai chatbot designers: Design conversational agents that adjust responses according to user preferences and expertise for better scientific support.
Authors
Siyang Wu, Weizhi Zhang, Chen Yang, Esther Xu Fei, Jorin Overwiening, Xu Pan, Jing Tang, Yixuan He, Kelvin C. Mo, Peiwen Li, Defu Cao, Xu Hu, Haixin Wang, Taoran Li, Haoming Li, Zheyuan Deng, Jingwen Zhang, Pengsong Zhang, Haichao Zhang, Guangze Luo, Yi Daniel Lu, Zihuai He, Chenxi Qiu, Dianzhuo Wang, Zhejian Peng, Shushu Liang, Wenyi Lu, Yifan Wang, Jintao Huang, Yijun Wang, Xiaomin Li, Yuexing Hao
Abstract
Scientific agents collaborate with researchers whose expertise and preferences differ, but benchmarks usually hold the collaborator fixed. We introduce PersonaScienceBench: 200 tasks from five benchmarks across four scientific domains. Simulated researchers provide context and feedback according to persona profiles, while task objectives and grading remain fixed. Agents encounter researcher preferences only through conversation. Across eighteen modelconfigurations, task performance, fact recovery, and self-repetition reveal different strengths. Detailed GPT 6 Astra analyses show distinct agent responses across researcher profiles; interaction traces illustrate targeted clar ification and workflow revision. These behaviors can differ even when final scores match. PersonaScienceBench therefore evaluates how agents collaborate with diverse researchers alongside whether they complete the task.