Safe AI advice must balance help and public risks
Fundamental Impossibility Results in AI Safety: Trading Off between Desiderata
Computer Science and Game TheoryArtificial Intelligence
Summary
Sometimes, when AI systems give advice, they need to help good users while avoiding harm if bad users see that advice. The authors show that if AI advice is public, there is a basic tradeoff: it can’t be both fully helpful and fully safe. They model this as a game between defenders, attackers, and AI, showing limits on what AI can do. This insight can guide how safer AI systems are designed.
What this means in practice
- •For security engineers: Design AI systems that provide advice while carefully managing risks from attackers who can view that advice.
- •For data privacy teams: Develop policies for sharing AI-generated information that balance transparency with preventing misuse by malicious actors.
A theory result. No direct application yet.
Authors
Ida Mattsson, Ratip Emin Berker, Emanuel Tewolde, Ioannis Anagnostides, Kiriaki Fragkia, Xinyu Liu, Alexander Heckett, Jiayuan Liu, Eden Hartman, Vojtěch Kovařík, Sven Neth, Maria-Florina Balcan, Tuomas Sandholm, Vincent Conitzer
Abstract
Is there a fundamental tradeoff between desiderata for safe AI systems, such as harmlessness and helpfulness? We answer by formalizing the interaction between AI systems and users as a game with three players: a defender and an attacker that are computationally or informationally bounded, and an AI that provides public advice and is less constrained. Our main result shows that there is a fundamental tradeoff in providing advice that helps the defender while advice remains public. We also consider whether the information that the AI shares can be manipulated to benefit only the defender and conclude with a discussion of how we can use these results in designing safer AI systems.