Agentx-model improves industrial recommendation experiments with dual-agent system
Advancing Model Research in AgentX: Long-Horizon Autonomy for Industrial Recommender Systems
Artificial IntelligenceInformation Retrieval
Summary
Improving recommendations in big companies means trying many experiments and learning from each one. The authors present AgentX-Model, a system where two automated agents work together: one suggests new ideas based on past research, and the other runs experiments and reports back results. This process helps build better recommendation models step-by-step, tracking what works and fixing problems. Their approach showed improvements in prediction accuracy and business metrics like ad spending efficiency and watch time.
What this means in practice
- •For industrial recommendation teams: Run connected experiments that automatically refine models to improve prediction accuracy and business impact over time.
- •For online advertising teams: Improve customer targeting and budget efficiency through better model research and evaluation driven by automated agents.
Authors
Shuang Yang, Zijie Zhuang, Changxin Lao, Pengbo Xu, Hanwen Xu, Yusheng Huang, Han Gao, Guanchen Wang, Tianbao Ma, Linxun Chen, Peilin Song, Xuming Wang, Chen Li, Fan Wu, Tao Wang, Zibo Zhao, Xiangyu Wu, An Liu, Fei Pan, Peng Jiang, Chen Yang, Zhaojie Liu, Wenwu Ou
Abstract
Sustaining industrial recommendation research requires using the results of one experiment to decide what to investigate next. We present AgentX-Model, the next generation of AgentX's model research framework, which connects proposal development and model experimentation within sandboxes defined by business inputs and prediction tasks. AgentX-Model adopts a dual-agent architecture comprising a Research Agent and a Model Agent. The Research Agent develops independently reviewed proposals from papers and experimental findings, while the Model Agent conducts multi-round investigations and returns code, measurements, and unresolved questions. Using the returned results, the Research Agent selects a starting implementation and formulates the next research question, allowing subsequent experiments to build on earlier findings. We organize this continuing research around four actions: Reproduce, Follow-up, Composition, and Diagnose. The first three actions drive routine research, while Diagnose acquires the evidence needed to choose a repair, including for issues raised by business feedback and online evaluation, such as prediction bias measured by PCOC. Across the production evaluation, 560 of 636 completed model-changing experiments recorded AUC above their business baselines. As research continued, some experiments recorded AUC above every comparable ancestor in their lineages. The five latest online A/B evaluations across different business settings reported gains including 10-15% in acquisition efficiency, 15-20% in target-segment advertising spend, and 0.3-0.8% in watch time; the watch-time model used approximately 10% fewer FLOPs and parameters. A dependency-aware historical-replay benchmark further evaluates research allocation, with initial results showing no consistent efficiency gain from more complex scheduling when agents already analyze and select concrete candidates.