AI tools enable online creation of non-consensual intimate images
AI-Generated Non-Consensual Intimate Imagery is Configured, Not Built: How General-Purpose AI Technology Enables Sexual Violence
Cryptography and Security
Summary
Creating sexual images of real people without their permission online is a growing problem made easier by general artificial intelligence tools. The authors studied publicly shared guides on how to make sexual AI images and found that the harmful, non-consensual kind can be made by simply adjusting settings rather than using special software. They explain how common AI image creation workflows can produce this content with little attention to ethics or harm. The paper also highlights that current defenses are limited because most AI models involved are openly available.
What this means in practice
- •For ai policy makers: Assess AI image models and products against practical attacker methods to comply with new EU regulations.
- •For ai safety engineers: Focus technical safeguards on limiting AI's generation of real people’s identity content to reduce non-consensual image creation.
Authors
Julia B. Kieserman, Ashley Phoebe Ishoel, Damon McCoy, Elissa M. Redmiles, Lucy Qin, Jaron Mink
Abstract
AI-generated non-consensual intimate imagery (AIG-NCII) is an increasingly prevalent form of online abuse, distinguished from AI-generated sexual content (AIG-SC) by the sexualized depiction of a real person's identity without their consent. To understand how offenders may benefit from the tools and resources shared across the broader AIG-SC community, we analyzed 139 R+ guides on CivitAI, a popular platform known to host a disproportionate amount of AIG-SC. We find that AIG-SC guides discuss how to create content generation workflows composed of sexual and identity concepts, the building blocks of AIG-NCII, including one guide that explicitly outlines an AIG-NCII workflow. We identify that ultimately AIG-NCII is enabled by configuration rather than a distinct set of technical tools. We map AIG-NCII threat models across basic and advanced AIG-SC workflows to demonstrate how multiple dual-use technical components can compose NCII outputs in an environment with little to no discussion of ethical implications or responsible use. We conclude by noting that, as most workflows rely on open-weight models, existing technical defenses against AIG-NCII are limited. However, our findings offer a starting point for evaluation of AI-based models and products against realistic attacker behavior required by new EU regulation and suggest that efforts to develop safeguards at the model- and product-levels should focus on generation of identity content.