Generative flow networks create diverse expert conversations for training ai
Beyond Mode Collapse: Generating Diverse Synthetic Expert Conversations via Generative Flow Networks
Artificial IntelligenceComputation and LanguageMachine Learning
Summary
Generating varied and realistic expert conversations is hard because usual AI methods tend to repeat common patterns. The authors used a new approach called Generative Flow Networks to create diverse synthetic dialogues by modeling key features of conversations. This method better captures different expert strategies and produces data that helps train AI models more effectively. They tested their method on tutoring and emotional support chats and showed it outperforms other popular techniques.
What this means in practice
- •For ai developers: Generate diverse synthetic expert dialogues to improve training data quality for adaptive AI models.
- •For customer support platforms: Create realistic varied conversation datasets to train and test support chatbots with better coverage of user scenarios.
Authors
Sumit Asthana, Michael Ion, Kevyn Collins Thompson
Abstract
High quality synthetic data is central to post training LLMs for adaptive AI applications that represent the diverse expert strategies and decisions in conversations. Prompting LLMs directly or conditioning them on end use scenarios yields low diversity data that collapses onto dominant modes. We propose a method to generate diverse high quality synthetic data using Generative Flow Networks (GFlowNets). We show that training GFlowNets to generate latent conversation structure using a Gaussian mixture density over key interaction features (e.g., confusion episode dynamics, scaffolding directive balance) enables sampling expert strategies in proportion to their prevalence in the training data. Across two structurally distinct domains, tutoring and emotional support dialogues, our GFlow based synthetic data generation approach offers a better balance of fidelity, mode coverage and authenticity than reinforcement-learning and end to end LLM baselines, without copying training data. Evaluated on three downstream outcome prediction tasks, classifiers trained on synthetic GFlowNet generated conversations provide a stronger training signal than competitive synthesis baselines.