Hierarchical cooperative networks improve exploration in generative models
Hierarchical Cooperative GFlowNets
Machine LearningArtificial Intelligence
Summary
Sampling diverse and interesting examples from complex spaces is hard because models tend to focus only on a few popular areas. The authors propose Hierarchical Cooperative GFlowNets, which let multiple models work together by dividing the search space dynamically. Higher-ranked models explore broadly while others focus on less-covered regions, encouraging diversity. Their method better covers all possibilities and produces samples that match the desired probabilities across many tasks.
What this means in practice
- •For combinatorial optimization engineers: Coordinate multiple samplers to explore diverse solutions, improving coverage of optimal options.
- •For biological sequence designers: Generate diverse biological sequences with probabilities aligned to target fitness, enhancing discovery of novel candidates.
Authors
Pedro Dall'Antonia, Juan Belieni, Ana Carolina Erthal, Saullo H. G. de Oliveira, Eliezer de Souza da Silva, Diego Mesquita
Abstract
Generative Flow Networks (GFlowNets) are trained to sample objects from compositional spaces with probability proportional to a reward function, but ineffective exploration can cause learned samplers to concentrate on a few high-reward regions. An intuitive remedy is to train multiple GFlowNets independently in parallel and combine their samples. However, this does not prevent multiple models from collapsing onto the same high-reward areas, leaving large regions of the state space unexplored. With this in mind, we propose Hierarchical Cooperative GFlowNets (HCG), an approach that dynamically partitions the search space to coordinate discovery without relying on hand-crafted boundaries. HCG imposes an ordering in which the highest-ranked model may explore the entire space, while lower-ranked models are progressively pushed toward regions not covered by higher-ranked ones. We provide a theoretical characterization of how specific learning objectives can shift probability mass away from designated areas, instantiating this mechanism within our architecture. A shared classifier coordinates the hierarchy by partitioning the search space, keeping the GFlowNets independently parameterized and enabling asynchronous training. Furthermore, we derive a normalizer-weighted aggregation rule for the ensemble and prove that it recovers the target reward-proportional distribution under the stated assumptions. Finally, experiments across multimodal distributions, sequence generation, combinatorial optimization, and biological sequence design show that HCG improves mode coverage and target approximation, translating increases in the number of coordinated samplers into broader, complementary exploration.