Fairness auditing made efficient for text to image AI models
Efficient Fairness Auditing Across Guidance Scales in Text-to-Image Diffusion Models via Causal Abstraction
Machine LearningComputer Vision and Pattern Recognition
Summary
Checking if AI that creates images from text is fair requires making lots of pictures with different settings, which takes a lot of computer power. The authors created a smarter way to do this by simplifying the AI process into higher-level parts and using a special model to predict fairness outcomes across different settings. This method saves time and still gives accurate fairness checks. They tested it on popular image-generating AI models to see how fair they are under different conditions.
What this means in practice
- •For ai model engineers: Accelerate fairness testing in text-to-image generation models without repeatedly generating large image sets.
- •For ai ethics teams: Perform efficient, scalable fairness audits on image generators across different guidance settings to ensure equitable outputs.
Authors
Nabila Tasfiha Rahman, Rajatsubhra Chakraborty, Depeng Xu, Lu Zhang
Abstract
Fairness auditing of text-to-image diffusion models often requires generating large numbers of images across sampling configurations, making comprehensive evaluation computationally expensive. We propose a causal-abstraction-based audit instrument for efficiently evaluating fairness under interventions on the classifier-free guidance scale. Given a fixed prompt and a target feature function, we represent the diffusion process as a low-level structural causal model and construct a corresponding high-level model over abstract denoising states. We characterize the projected causal structure, establish identifiability of the fairness-relevant interventional query, and provide sufficient conditions under which the high-level model preserves this query. A probabilistic transformer implements the high-level model as an amortized predictor of target-feature distributions across guidance scales. Experiments evaluate distributional fidelity, fairness-query accuracy, and computational efficiency. We present two auditing demonstrations: one using standard Stable Diffusion 1.5 and another using StayFair, a fairness-enhanced Stable Diffusion model, to examine their behavior across guidance scales.