Algorithmic harms rise with generative models in recommendation systems
Algorithmic Harms Associated with Generative Model-Augmented Recommendation Systems
Machine LearningArtificial IntelligenceComputation and LanguageInformation Retrieval
Summary
When recommendation systems use generative AI models, new types of problems can appear beyond what people have already studied. The authors show that these problems come from how the system inputs and the AI's assumptions don't always match the goals of those building the system. They introduce a new way to categorize these harms and explain how certain harmful patterns happen, which can help detect and fix these issues.
What this means in practice
- •For machine learning engineers: Detect and address new algorithmic harms caused by generative models in recommendation systems by using the expanded harm taxonomy and causal analysis.
- •For data governance teams: Develop better harm mitigation policies for generative model-augmented platforms by understanding novel causal drivers and endogenous harms described in the taxonomy.
A position paper. It proposes an approach and reports no results.
Authors
Christine Herlihy, Xumei Xi, Shloka Desai, Kevin Bannerman Hutchful, Pedro Silva
Abstract
In this work, we consider algorithmic harms that may arise as generative models are incorporated into machine learning platforms. We argue that existing harm taxonomies and threat models require extension to (1) address novel causal drivers of well-studied representational and quality-of-service harms; and (2) anticipate and mitigate endogenous harms, such as sanitization, which may arise when system inputs are misaligned with the system designer's objectives, or the generative model's inductive priors. To this end, we introduce an expanded taxonomy of algorithmic harms associated with the use of generative models in non-conversational recommendation systems. In addition, we offer a causal analysis of how problematic subsets of the (input, output) joint distribution can arise, in an effort to inform harms detection and mitigation efforts.