URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment
2026-07-06 • Machine Learning
Machine LearningArtificial IntelligenceComputational Engineering, Finance, and ScienceComputation and Language
AI summaryⓘ
The authors created a new method called URSA to better compare how well different computer models can plan chemical synthesis routes for making molecules, especially in drug discovery. Their method checks not only if the plans use available starting materials but also if the chemical steps make sense like an expert chemist would judge. They tested both specialized computer models and large language models on unknown target molecules and found that language models can help with overall planning but are less reliable than specialized models at detailed synthesis tasks.
Synthesis planningRetrosynthesisDrug discoveryDeep learningLarge language modelsBenchmarkingChemical plausibilitySynthetic routesURSA evaluation frameworkReaction pathways
Authors
Bogdan Zagribelnyy, Ivan Ilin, Nikita Bondarev, Anton Morgunov, Arkadii Lin, Maksim Kuznetsov, Rim Shayakhmetov, Vladimir Aladinskiy, Alex Aliper, Alex Zhavoronkov
Abstract
Synthesis planning aiming to find pathways of reactions for a target molecule is one of the most important and challenging tasks in drug discovery. Recent progress has produced both specialized deep-learning retrosynthesis systems and general-purpose large language models, but objective comparison remains difficult due to the lack of flexible, chemically interpretable benchmarking protocols. In the current study, we are introducing the URSA (Utilitarian RetroSynthesis Assessment) evaluation framework that provides the opportunity to benchmark the synthetic routes not only from a formal perspective, such as convergence to commercially available starting materials, but also from a chemical plausibility perspective, mimicking the way expert chemists evaluate the reactions and routes. The study covers a comprehensive evaluation of both conventional end-to-end retrosynthesis solutions and LLMs for the synthesis planning task on a set of novel, diverse target molecules with undisclosed synthetic routes, which represent realistic tasks in the daily drug design routine. We find that while LLMs can support high-level strategic planning, they currently underperform specialized retrosynthesis models in reliably solving synthesis planning tasks.