Multi-agent flow matching generates objects meeting hard constraints
Multi-Agent Flow Matching with Decoupled Generative Guidance
Machine LearningMultiagent SystemsRobotics
Summary
Creating multiple things at once often means they need to work together and follow strict rules, which is tricky. The authors introduce a method called DeGG-Flow that helps each part decide how to act without depending on others simultaneously. Their approach can handle both rules that involve the whole group and rules that only concern a few nearby parts. They show that using this method leads to generating objects or actions that always follow those strict rules, even when there are more parts than before.
What this means in practice
- •For robotics engineers: Generate coordinated multi-robot plans that satisfy spatial constraints without needing simultaneous cross-communication of guidance inputs.
- •For scene designers: Create multi-object scenes meeting interaction affordances automatically without trial and error during object placement.
Authors
Ruoyu Lin, Magnus Egerstedt, Fabio Pasqualetti
Abstract
Generative modeling is widely used for producing diverse objects from complex, multimodal distributions. However, its expressivity does not, in general, come with formal guarantees that the generated objects satisfy hard constraints or requirements. In multi-agent generation, this problem becomes more challenging because a hard requirement can depend on multiple agents, while each agent may need to determine its own guidance input without relying on the simultaneously computed guidance inputs of other agents. To this end, we introduce DeGG-Flow, a general framework for multi-agent flow matching with decoupled generative guidance. By representing the generative process as a control-affine dynamical system, we develop guidance conditions for two classes of coupled requirements: shared requirements whose satisfaction depends on multiple agents together, and private requirements associated with each individual agent dependent on its neighbors. For both classes, we establish feasibility conditions and finite-horizon convergence guarantees. We further derive a Wasserstein bound that characterizes the distributional deviation induced by the guidance. We demonstrate DeGG-Flow on multi-robot collaboration for crossing a spatial gap by reconfiguring the environment, and on multi-object scene generation with affordance requirements. Across both applications, DeGG-Flow directly generates objects that satisfy all corresponding hard requirements, including at team sizes unseen during training.