Acceptable misalignment guides safe development of autonomous robots
Acceptable Misalignment
Artificial IntelligenceRobotics
Summary
Designing robots that learn on their own is hard because we don't know exactly how they'll act or what they'll learn. The authors explain that perfectly matching robot behavior with human values is impossible, so mistakes will happen. Instead of trying to stop all mistakes, they suggest focusing on when those mistakes can be considered okay or acceptable. They introduce ideas and frameworks to help create alignment methods that handle inevitable misalignment in a safe and ethical way.
What this means in practice
- •For robot developers: Develop alignment systems that manage and tolerate acceptable levels of robot misbehavior during learning and deployment.
- •For ai safety engineers: Implement frameworks that shift focus from perfect alignment to controlling the risks of autonomous system misalignment incidents.
A position paper. It proposes an approach and reports no results.
Authors
Maud van Lier, Ludovica Marinucci, Vieri Giuliano Santucci, Raja Chatila, Stéphane Doncieux, Émiland Garrabé, Richard Duro
Abstract
There is a specific challenge designers face in developing autonomous robots capable of open-ended learning. This challenge is how one can develop such systems in an ethically sound way when there are many unknowns during design time about their manner of deployment, what they can and will learn, and how these systems will act on the basis of what they have learned. One response to this challenge is to align, as far as possible, the activities and goals of autonomous robots with human values, raising the question of how to make this alignment possible, a question known as the value alignment problem. We argue that the value alignment problem faces a number of challenges that show that full alignment is both theoretically and practically impossible, meaning that misalignment incidents will keep occurring. This undermines current alignment approaches, as the attempt to maximize alignment cannot properly deal with the possibility that the risk of misalignment that remains after alignment methods have been implemented might still be unacceptable to society. To overcome this difficulty, we argue that a shift in perspective is required from a focus on the maximization of alignment to the conditions under which misalignment incidents become acceptable. In this paper, we explore what this shift in perspective entails. We introduce the concept of acceptable misalignment, formulate normative requirements for alignment mechanisms aimed at bringing misalignment incidents to an acceptable level, and we conclude by presenting a new methodological framework that consists of alignment mechanisms that can meet these requirements.