One-step flows enable fast generation of categorical data

ReSFlow: Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation

Machine LearningArtificial Intelligence

Summary

Generating complex categorical data like text or images often requires many slow, gradual steps. The authors propose a new method called ReSFlow that learns the most direct paths—called geodesics—between simple starting points and target data on a special curved space of probabilities. This lets the model create realistic discrete data in just one step, making the process much faster and computationally cheaper. Their tests on various tasks show ReSFlow produces high-quality results quickly.

What this means in practice

  • For machine learning engineers: Generate discrete data such as text or images in a single inference step, reducing latency and computation costs in production ML pipelines.
  • For bioinformatics analysts: Produce complex biological sequence data in one step by learning geodesic flows on statistical manifolds, speeding simulation and analysis workflows.

Authors

Xiaohuan Jia, Renzhe Xu, Xiao Wang, Jiayun Wu, Shaohua Fan

Abstract

Generative modeling for discrete data has been significantly advanced by learning continuous vector fields on the statistical manifolds to transport simple priors to target discrete distributions. However, because these continuous-time flows are not constrained to follow geodesics on the statistical manifold, thereby requiring multi-step numerical integration for simulation and leading to substantial computational costs during inference. In this work, drawing inspiration from the "straight and fast" properties of Rectified Flow in continuous Euclidean space, we propose the Rectified Statistical Flow (ReSFlow) model, which enables one-step categorical generation on statistical manifolds by explicitly learning geodesic trajectories that adhere to the intrinsic Riemannian geometry. By rectifying the flow on the statistical manifold, we theoretically guarantee that trajectories between prior and target distributions approximate geodesics. This rectification significantly reduces the transport cost by aligning the learned trajectories directly along geodesics on the statistical manifold, allowing the model to map pure noise to target discrete data in a single step. Through extensive experiments on diverse real-world generative tasks, ranging from image and text to biological domains, we demonstrate ReSFlow's superior one-step generation quality, which truly embodies geodesically optimal and rapid categorical generation. Our code is available at https://github.com/xiaohuanjia/ReSFlow.