MARCO improves iterative molecule editing with multi-round reinforcement
MARCO: Multi-Round Agentic Reinforcement for Conditional Molecular Optimization
Machine LearningComputation and Language
Summary
Designing better molecules often involves making step-by-step changes and checking if they meet certain goals, like being similar to the original molecule while improving its properties. The authors propose MARCO, a new method that trains AI models to improve molecules over multiple rounds by using feedback after each edit. This approach better balances creating valid, improved, and similar molecules compared to methods that try to do everything in one step. Tests show MARCO’s method works well across different instructions and molecule goals.
What this means in practice
- •For pharmaceutical development teams: Enhance drug candidate design by iteratively refining molecules with feedback for better efficacy and similarity to target structures.
- •For chemical materials researchers: Improve material molecules by using multi-round AI-guided editing that balances desired properties and molecular constraints over several iterations.
Authors
Shicheng Fang, Yuxin Wang, Zhuo Yang, Xiaohu Xu, Jiahao Lu, Chuanyuan Tan, Tong Zhu, Yining Zheng, Xipeng Qiu
Abstract
Molecular optimization is inherently iterative: a candidate is proposed, evaluated against several objectives, and revised while preserving a relationship to the source molecule. Most instruction-following models instead emit one edited molecule, forcing validity, property improvement, and similarity control into a single response. We introduce MARCO, an evaluator-grounded reinforcement-learning framework that trains molecular editors on bounded proposal--feedback--revision trajectories. MARCO aggregates shaped turn rewards into an undiscounted trajectory return for group-relative policy optimization. We evaluate two consequences of this training: Same-1 tests the trained policy under a one-response budget, while Same-5 tests whether the same policy can use verifier feedback when up to five responses are available. Across the three-objective MuMOInstruct benchmark, three Qwen backbones, and seen/unseen instruction splits, SFT-initialized MARCO obtains the highest product of property success rate and similarity in every reported primary setting. Same-5 further improves the observed score under the tested budget, while four-objective and public-checkpoint experiments test transfer across constraint sets and initialization regimes.