Reinforcement learning improves RNA design for better structure and function

MORAD: Reinforcement Learning with Multi-Objective Rewards for RNA Inverse Design

Machine Learning

Summary

Designing RNA sequences that fold into desired 3D shapes is a complex challenge important for biology and medicine. The authors created MORAD, a method that trains a single AI model to design RNA sequences matching various 3D backbone shapes by considering multiple factors at once, like structure and chemical properties. This approach outperforms previous methods that optimized only one feature at a time, yielding more accurate and reliable RNA designs. MORAD works by combining different measurements into a balanced score, allowing the model to learn how to create better RNA molecules.

What this means in practice

  • For biotech developers: Generate RNA sequences customized to specific 3D shapes for use in therapeutics and synthetic biology applications.$Commercial implications: Enables biotech companies to produce tailored RNA molecules for drug design and gene regulation tools, improving product development.
  • For computational biology teams: Improve RNA design workflows by integrating an AI model that balances multiple structural and chemical objectives for greater accuracy.

Authors

Jixin Tang, Ji Guo, Jun Zhao

Abstract

We introduce MORAD, an online reinforcement learning framework for backbone-conditioned RNA diffusion models that trains a single shared policy across diverse target backbones. Structure-conditioned RNA inverse design seeks nucleotide sequences compatible with a target three-dimensional backbone, yet existing reinforcement learning approaches either optimize a separate policy for each target backbone or rely on offline preference optimization, leaving online post-training of a reusable backbone-conditioned model largely unexplored. MORAD combines backbone-conditioned group sampling, group-relative reward normalization, and a forward-process diffusion objective. Beyond existing RL rewards that primarily target tertiary structural quality, MORAD jointly optimizes six measurements spanning tertiary geometry, base pairing, ensemble behavior, and sequence composition. To aggregate these heterogeneous objectives despite differences in scale and optimization direction, MORAD maps each measurement to a bounded desirability score and combines them with a weighted geometric mean. Compared with a tertiary-structure-only reward, this multi-objective formulation improves 10 of the 11 evaluation metrics that have a preferred direction. Applied to pretrained RIDE, MORAD improves GDT-TS by 16.0% (from 0.3300 to 0.3827) and pairing MCC by 19.4% (from 0.6110 to 0.7295), while reducing normalized ensemble defect by 27.5% (from 0.3906 to 0.2830). On the test set, MORAD outperforms RiboDiffusion, gRNAde, and RDesign across tertiary structural accuracy, secondary-structure pairing, and thermodynamic quality, establishing a new state of the art in structure-conditioned RNA inverse design.