Discrete diffusion adaptation improves protein and DNA sequence design

D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform

Machine LearningArtificial Intelligence

Summary

Designing useful biological sequences like DNA or proteins is hard because many possible sequences exist. The authors present a new way to tweak existing models at sampling time without extra training, helping create sequences that score better on desired properties. Their method samples different sequence endings, evaluates them using rewards, and chooses better options, leading to more effective sequences. This approach works well on tasks like designing DNA that boosts cell activity and on folding proteins correctly.

What this means in practice

  • For biotech researchers: Generate synthetic DNA sequences with improved regulatory activity for gene expression experiments without retraining models.
  • For protein engineers: Produce candidate protein backbones that fold correctly, enhancing the design of novel proteins in biomedical research.

Authors

Jieke Wu, Qijie Zhu, Weimin Wu, Zeqi Ye, Minshuo Chen, Han Liu

Abstract

We propose D-DOIT (Discrete Doob-Oriented Inference-time Transformation), a training-free and efficient adaptation method for discrete diffusion models with generic rewards. D-DOIT formulates adaptation as sampling from a reward-tilted target distribution and realizes this transport through Doob's h-transform of the discrete diffusion reverse kernel, using only reward values rather than reward gradients. Unlike continuous diffusion, masked discrete diffusion samples categorical token-reveal transitions rather than continuous state updates. D-DOIT derives the corresponding discrete Doob's h-transform, which guides sampling by reweighting reverse transition probabilities instead of adding a drift correction. To make this transformation practical, D-DOIT avoids expensive future rollouts. At each guided step, D-DOIT samples candidate next states, uses the model prediction head to complete each candidate into a clean sequence, evaluates each completion with the reward oracle, and resamples the next state with probabilities proportional to the rewards. An optional late-stage best-of-K refinement further improves sample quality by branching trajectories only near the end of denoising, avoiding the $K$-fold cost over the full trajectory. Empirically, across regulatory DNA design and protein inverse folding benchmarks, D-DOIT outperforms training-free guidance baselines. It improves enhancer activity and cell-type specificity while preserving sequence naturalness, and achieves the highest success rate in protein inverse folding.