Continuous-time generative models create smooth data evolution paths
Walking the Score Manifold: Continuous-time Generative Dynamics on Learned Data Manifolds
Machine LearningArtificial Intelligence
Summary
Generating time-based data usually happens step-by-step at fixed moments, but this paper shows a way to create data that changes continuously over time. The authors use existing models that understand the data’s shape to guide smooth transitions between points, allowing creation at any moment, even those not originally recorded. Their method also learns to keep these transitions stable and can model multiple possible futures. They test their approach on videos, scientific simulations, and molecular movements, showing it works well for high-quality, time-continuous generation.
What this means in practice
- •For video processing teams: Generate high-quality slow-motion or frame-interpolated videos without needing densely sampled data.
- •For molecular simulation engineers: Produce realistic continuous-time molecular dynamics trajectories with fewer discrete steps from training data.
Authors
Jan Tauberschmidt, Brian B. Moser, Stanislav Frolov, Andreas Dengel, Andrew B. Duncan, Sebastian J. Vollmer
Abstract
Generative modeling of time-dependent data is typically formulated on a discrete temporal grid, restricting supervision to the observed timestamps in the training data. We instead frame generation as continuous-time evolution on a learned data manifold. To this end, we leverage pretrained score-based models as geometric priors and learn a vector field that evolves data along score-induced interpolation paths. Because these dynamics follow transitions that respect the geometry learned by the score model, they support generation at arbitrary timestamps and temporal super-resolution beyond the discretization of the training data. Moreover, this geometric formulation allows us to train the vector field simulation-free through a regression objective. To improve long-horizon rollout robustness, we introduce an objective that promotes path-relative transverse exponential stability. While motivated by stability theory, it admits a practical interpretation as denoising score matching transverse to the interpolation path. Further, we extend the framework to a probabilistic setting that models a distribution over plausible future trajectories. We demonstrate the method on natural video and scientific dynamical data, including temporal super-resolution, PDE-based spatiotemporal fields, and molecular dynamics. Our results show that score-based priors provide a strong foundation for learning stochastic continuous-time generative dynamics.