Physics informed generative models improved with data anchored penalties
Characterizing and Mitigating Distributional Bias in Physics-Informed Generative Models
Machine Learning
Summary
Physics-informed generative models try to create data that follow physical rules, but making these rules more strict doesn't always make the data more accurate. The authors found that models can favor certain states just because they have lower sensitivity to errors, even if those states aren't actually where real data lie. To fix this, they propose a penalty method that links errors directly to the original training data characteristics, which helps the model stay more faithful to real-world data. They tested their method on synthetic systems, partial differential equation systems, and molecular transition states, showing better results.
What this means in practice
- •For computational physicists: Generate simulation data that respect physics constraints more accurately without drifting to unrealistic states.
- •For drug discovery teams: Improve generative models for molecular transition states to better suggest chemically plausible intermediate structures.
Authors
Tanja Bien, Samir Darouich, Lluís Pastor-Pérez, Vinh Tong, Loay Mualem, Mathias Niepert
Abstract
Physics-informed generative models augment a data-fitting objective with residual penalties that encourage physical constraints to hold in generated samples. However, improving constraint satisfaction need not improve fidelity to the data distribution. We study this conflict in residual-regularized diffusion models and identify two possible consequences. First, the same prediction error can incur different residual costs near different feasible states, preferring states with lower residual sensitivity (avoidance). Second, this preference can extend to feasible states outside the data support, where the residual also vanishes (leakage). Motivated by this analysis, we propose a data-anchored penalty that weights prediction errors by the residual Jacobian at the clean training sample and normalizes the resulting magnitude. This removes the incentive to relocate to less sensitive states without changing the model architecture or sampling procedure. Experiments on a controlled synthetic system, three PDE systems, and molecular transition-state generation with flow matching demonstrate the benefits of anchoring. Code is available at https://github.com/TanjaBien/anchored_diffusion.